-- Logs begin at Fri 2022-06-24 08:34:59 UTC, end at Fri 2022-06-24 10:15:41 UTC. -- Jun 24 09:04:22.908773 dv-cinder-845990-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 09:04:22.920594 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 09:04:22 2022] *** Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: nodename: dv-cinder-845990-4-iscsi Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: machine: x86_64 Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: clock source: unix Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: pcre jit disabled Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: detected number of CPU cores: 4 Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: current working directory: / Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: detected binary path: /usr/bin/uwsgi-core Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: your processes number limit is 47680 Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: your memory page size is 4096 bytes Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: detected max file descriptor number: 2048 Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: lock engine: pthread robust mutexes Jun 24 09:04:22.921137 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: thunder lock: enabled Jun 24 09:04:22.921839 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 09:04:22.921839 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 09:04:22.965583 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: Python main interpreter initialized at 0x55d3e10729c0 Jun 24 09:04:22.965583 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: python threads support enabled Jun 24 09:04:22.965583 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: your server socket listen backlog is limited to 100 connections Jun 24 09:04:22.965583 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: your mercy for graceful operations on workers is 90 seconds Jun 24 09:04:22.965939 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 09:04:22.966017 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: *** Operational MODE: preforking *** Jun 24 09:04:22.966270 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: *** uWSGI is running in multiple interpreter mode *** Jun 24 09:04:22.966270 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: spawned uWSGI master process (pid: 80147) Jun 24 09:04:22.966457 dv-cinder-845990-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 09:04:22.967843 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: spawned uWSGI worker 1 (pid: 80156, cores: 1) Jun 24 09:04:22.967843 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: spawned uWSGI worker 2 (pid: 80157, cores: 1) Jun 24 09:04:22.967843 dv-cinder-845990-4-iscsi devstack@keystone.service[80147]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 09:04:28.456511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:04:28.456511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: from cryptography.utils import int_from_bytes Jun 24 09:04:28.457315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:04:28.457315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: from cryptography.utils import int_from_bytes Jun 24 09:04:28.458190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:04:28.458190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: from cryptography.utils import int_from_bytes Jun 24 09:04:28.458635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:04:28.458635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: from cryptography.utils import int_from_bytes Jun 24 09:04:28.505672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 09:04:28.505861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 09:04:28.506056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] command line args: [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 09:04:28.506332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 09:04:28.506513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ================================================================================ {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 09:04:28.506605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 09:04:28.506769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] admin_token = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.506855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 09:04:28.506959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] command line args: [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 09:04:28.507037 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] config_dir = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.507191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.507278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 09:04:28.507374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] config_source = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.507451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ================================================================================ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 09:04:28.507555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] debug = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.507633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] admin_token = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.507760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config_dir = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] config_source = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] debug = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.508798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509124 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] list_limit = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] log_config_append = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] log_dir = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.509932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] log_file = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.510052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_config_append = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.510781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.510869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.510869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_dir = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.511940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.512975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.513949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.514199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.514279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.514385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] publish_errors = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.514621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.514739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.514849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] publish_errors = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.514938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] standard_threads = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.515929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] standard_threads = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] use_journal = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] use_json = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] use_stderr = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.516987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_journal = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.517064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] use_syslog = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.517144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_json = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.517262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.517340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_stderr = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.517523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.517607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] use_syslog = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.517770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.517854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:04:28.518025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518261 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.518994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.519954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.520940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.521983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.522965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.523927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.524869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.525993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.526944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.527927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.528874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.529953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.530902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531354 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.531871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.532996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.533967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534772 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.534934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.535984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.536921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.537871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.538984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.539988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.540938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.541085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.541165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.541845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.541931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] auth.external = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.542898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.external = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] auth.password = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.543878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] auth.token = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544261 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.password = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] auth.token = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544595 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.544995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.545957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] credential.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.546988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.547944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.548125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.548206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.548377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.548499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.548636 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.548721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.548933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.549967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.550955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.551914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.552921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.553998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.554947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.555905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.556075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.556163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.556326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.556406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.556623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.556707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.557153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.557791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.557900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.557984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.558975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.559947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.560916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561317 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.561897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.562988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.563934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.564912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.565935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566165 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.566863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.567868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.568019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.568099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.568264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.568352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.568500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.568602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569124 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.569981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.570952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.571918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.572050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.572134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.572292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.572371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.572569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.573154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.573317 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.573398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.573565 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.573644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.573806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.573886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.574988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.575933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.576933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.577913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] role.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] role.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.578993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.579991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.580158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.580240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.580407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.580497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.580667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.580750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.580920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.581941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.582075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.582186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.582315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.582569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.582648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.582807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.582895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.583866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.584851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.585851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.586964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.587987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.588136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.588812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.588982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589792 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.589953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.590973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.connection = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.591941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.592985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.593928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.594870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.595964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.596045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.596127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80156) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 09:04:28.596245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.596482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.596753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:04:28.596922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80157) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 09:04:28.626438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.626664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.626896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.628226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.628489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.628774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.635944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.637909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.664111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.664343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.665900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.666133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.806113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.807787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.855476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.855712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.856740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.856978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.919788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.919969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.920960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.921088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.946315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.946478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.946592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.946757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.946894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.947023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.947157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.947299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.947432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.947607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.948234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.948363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.948494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.948494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.959144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.959625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.972015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.972247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.972472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.972643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.972774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.972902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973028 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973165 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80156) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.973950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80157) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:04:28.990638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.990848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:04:28.991100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.991231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.991370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:04:28.991498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:04:28.991630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.991754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.991885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:04:28.992023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.992154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.992298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.992429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.992583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.992712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:28.992860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.003678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.003954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.004222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.004496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.004673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.004913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.005175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.005447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.013417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.015167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.016755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.017033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.017378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.017784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.017922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.018084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.018346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.018614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.018764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.018907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.019057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.019199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.019339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.019478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.019626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.019775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.020005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.020264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.020530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.020853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.021143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.021421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.046119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.047761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.048040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.048663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.050280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.050554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.053362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.053761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.054043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.055784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.056217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.056446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.062257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.062540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.064599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.064961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.067080 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.067364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.067648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.067929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.068203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.068475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.068769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.069036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.070340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.070631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.070913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.071191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.071466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.071736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.072004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.072270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.088423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.088728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.091604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.091882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.096800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.098499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.100542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.101075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.103984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.104387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.106469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.109482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.110109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.113132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.113786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.114082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.114364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.114661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.114942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.115218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.116837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.117137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.117426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.117707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.117984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.118270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.134842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.135744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.136014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.136283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.138551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.138838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.139104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.139371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.144173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.145916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.146186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.147335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.149079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.149356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.149451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.149706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.149990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.150232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.150520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.152680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.152986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.153286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.153620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.153904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.163852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.164130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.164424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.164727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.166421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.166707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.166997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.167218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.167502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.167793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.168073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.169750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.170053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.170354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.185714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.189058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.189152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.192349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.192452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.194097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.195719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.195815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.196012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.196295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.197488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.199694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.199976 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.200255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.204237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.205896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.209159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.209329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.209473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.209819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.210096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.211613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.214955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.215242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.215634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.215915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.219306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.219594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.219873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.225108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.225395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.225679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.227756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80157) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.228036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.228312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.228602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.228902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.229286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.229577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.229833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.230113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.230364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.230629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.230902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.231163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.231427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80157) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.233648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80156) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:04:29.233931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.234210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.234484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.234755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.235134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.235421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.235689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.235957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.236235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.236502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.236799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.237064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.237344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80156) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:04:29.272046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d3e10729c0 pid: 80157 (default app) Jun 24 09:04:29.275476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31672ca-cff6-4c1b-b4d1-51925f19a300 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:29.275797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31672ca-cff6-4c1b-b4d1-51925f19a300 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:29.276075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31672ca-cff6-4c1b-b4d1-51925f19a300 None None] PATH_INFO: `/v3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:29.276269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d3e10729c0 pid: 80156 (default app) Jun 24 09:04:29.280230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Fri Jun 24 09:04:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:37.009074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db67317-702b-43fd-846a-d155f609944d None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:37.009659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db67317-702b-43fd-846a-d155f609944d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:37.009659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db67317-702b-43fd-846a-d155f609944d None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:37.014573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:04:37.020269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:37.020743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:37.020863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:37.091280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: INFO dbcounter [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] Registered counter for database keystone Jun 24 09:04:37.112203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG oslo_db.sqlalchemy.engines [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80157) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 09:04:37.118510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writer thread running {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 09:04:37.299128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG passlib.handlers.bcrypt [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80157) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 09:04:37.299553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG passlib.handlers.bcrypt [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80157) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 09:04:37.353768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:37.355372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:37.480238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 09:04:37.620015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db67317-702b-43fd-846a-d155f609944d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:37.690140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: INFO dbcounter [None req-5db67317-702b-43fd-846a-d155f609944d None None] Registered counter for database keystone Jun 24 09:04:37.708874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG oslo_db.sqlalchemy.engines [None req-5db67317-702b-43fd-846a-d155f609944d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80156) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 09:04:37.715096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writer thread running {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 09:04:37.835249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db67317-702b-43fd-846a-d155f609944d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:37.839460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635005d-3886-47c6-a166-ef7923f034eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:37.840119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635005d-3886-47c6-a166-ef7923f034eb admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:37.840411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635005d-3886-47c6-a166-ef7923f034eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:37.840736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635005d-3886-47c6-a166-ef7923f034eb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:37.848816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635005d-3886-47c6-a166-ef7923f034eb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:37.908147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9635005d-3886-47c6-a166-ef7923f034eb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 09:04:37.908415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635005d-3886-47c6-a166-ef7923f034eb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:37.919105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Fri Jun 24 09:04:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 09:04:37.925207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:37.939804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8a771-d404-4c44-bdb1-17710e61a873 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:37.944221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a52133-c00a-4230-829b-58583471f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:37.945355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a52133-c00a-4230-829b-58583471f85b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:37.945647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a52133-c00a-4230-829b-58583471f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:37.945926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a52133-c00a-4230-829b-58583471f85b admin admin] PATH_INFO: `/v3/projects/ccae62deda0a4d329baa9cb863e45e2d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:37.949988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a52133-c00a-4230-829b-58583471f85b admin admin] RBAC: Authorizing `identity:get_project(project_id=ccae62deda0a4d329baa9cb863e45e2d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:38.001005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING keystone.common.rbac_enforcer.enforcer [None req-61a52133-c00a-4230-829b-58583471f85b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 09:04:38.001348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a52133-c00a-4230-829b-58583471f85b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:38.005868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Fri Jun 24 09:04:37 2022] GET /identity/v3/projects/ccae62deda0a4d329baa9cb863e45e2d => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:04:40.002641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c36d9-a835-412c-943f-0f90478890cd None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:40.003185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c36d9-a835-412c-943f-0f90478890cd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:40.003185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c36d9-a835-412c-943f-0f90478890cd None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:40.007296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:04:40.012766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f805-868f-4072-a580-a0da2edae30a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:40.013189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f805-868f-4072-a580-a0da2edae30a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:40.013329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982f805-868f-4072-a580-a0da2edae30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:40.031947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4982f805-868f-4072-a580-a0da2edae30a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:40.033521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4982f805-868f-4072-a580-a0da2edae30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:40.041875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 09:04:40.064661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c36d9-a835-412c-943f-0f90478890cd None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:40.076664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c36d9-a835-412c-943f-0f90478890cd None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:40.080791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e907854-ef1f-412e-8ea9-9ce2b0639ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:40.081426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907854-ef1f-412e-8ea9-9ce2b0639ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:40.081712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907854-ef1f-412e-8ea9-9ce2b0639ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:40.081993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907854-ef1f-412e-8ea9-9ce2b0639ceb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:40.089884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e907854-ef1f-412e-8ea9-9ce2b0639ceb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:40.091637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e907854-ef1f-412e-8ea9-9ce2b0639ceb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:40.100315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Fri Jun 24 09:04:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:04:40.106142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f805-868f-4072-a580-a0da2edae30a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:40.115091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982f805-868f-4072-a580-a0da2edae30a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:40.120024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72a487-fa24-402e-8f32-d97b48fe1c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:40.120902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72a487-fa24-402e-8f32-d97b48fe1c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:40.121185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72a487-fa24-402e-8f32-d97b48fe1c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:40.121457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72a487-fa24-402e-8f32-d97b48fe1c3e admin admin] PATH_INFO: `/v3/users/c06586f3b93746d589ca131fe0aad76b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:40.122829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72a487-fa24-402e-8f32-d97b48fe1c3e admin admin] RBAC: Authorizing `identity:get_user(user_id=c06586f3b93746d589ca131fe0aad76b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:40.124665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72a487-fa24-402e-8f32-d97b48fe1c3e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:40.126204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Fri Jun 24 09:04:40 2022] GET /identity/v3/users/c06586f3b93746d589ca131fe0aad76b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:42.214237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81239cbf-2711-4fa9-ad7e-64f5c6fded55 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.214772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81239cbf-2711-4fa9-ad7e-64f5c6fded55 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.214772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81239cbf-2711-4fa9-ad7e-64f5c6fded55 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.216139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:42.221220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e284f48-5dda-4983-8bc7-38b51307afaf None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.221638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e284f48-5dda-4983-8bc7-38b51307afaf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.221918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e284f48-5dda-4983-8bc7-38b51307afaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.238356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8e284f48-5dda-4983-8bc7-38b51307afaf None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:42.239536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8e284f48-5dda-4983-8bc7-38b51307afaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:42.251633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1605d4-dede-4b89-a05c-c0cdd3f88ec3 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.251921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1605d4-dede-4b89-a05c-c0cdd3f88ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.252205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1605d4-dede-4b89-a05c-c0cdd3f88ec3 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.253593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:04:42.258332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.258626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.258914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.298818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG passlib.handlers.bcrypt [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80156) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 09:04:42.299162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG passlib.handlers.bcrypt [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80156) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 09:04:42.321818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:04:42.345696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:42.346094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e284f48-5dda-4983-8bc7-38b51307afaf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:42.347285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:42.369974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:42.446102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:04:42.450559 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e284f48-5dda-4983-8bc7-38b51307afaf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:42.468850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:42.483452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:42.484256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.484541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.484860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.497116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:04:42.508214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:42.509156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:04:42.510852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:04:42.516227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:42.568004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00835a-f5ef-436f-bc74-d911524acbe4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:42.570222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:42.598939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:42.607469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:42.608368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.608690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.608972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.618769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef53f-456c-401d-ace4-cb75a63e24e9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:42.654140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:42.654806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.655094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.655374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.656131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:42.656448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:42.657990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:42.669637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:04:42.675182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:42.677554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:42.679063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:42.773528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad91e9-1ca9-41a9-a420-e8bcd7af2571 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:42.777899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:04:42.786280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:42.789178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:42.810672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:42.811550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.811852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.812129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] PATH_INFO: `/v3/users/c06586f3b93746d589ca131fe0aad76b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.813655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] RBAC: Authorizing `identity:get_user(user_id=c06586f3b93746d589ca131fe0aad76b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:42.815148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:42.816816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/users/c06586f3b93746d589ca131fe0aad76b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:04:42.822854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:42.836952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:42.886057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcfc61-978c-4f6d-99c7-367368931ace None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:42.920715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:42.921507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.921792 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.922068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.922204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d94db9-40da-40b6-9635-049b523cc0d6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:42.943479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:42.945024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:42.957516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:42.958307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:42.958594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:42.958870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:42.960047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:42.961538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:42.963140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:04:42.965551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:42.966361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:42.967165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:04:42.967165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:04:42.967165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:04:42.967165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:04:42.967165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:42.968915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:42.972216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:43.031834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:43.053668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:43.074498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c62804-5796-4287-b2a0-893f0f3805e9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:43.077442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359e4e8-b42a-4920-8b68-c53d9bde5ff0 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:43.108931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032e821-8037-4c94-aac4-2c2bbbdc53d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:43.109589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032e821-8037-4c94-aac4-2c2bbbdc53d3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:43.109875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032e821-8037-4c94-aac4-2c2bbbdc53d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:43.110150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032e821-8037-4c94-aac4-2c2bbbdc53d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:43.110983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032e821-8037-4c94-aac4-2c2bbbdc53d3 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.domain.id=default, user.id=c06586f3b93746d589ca131fe0aad76b) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:43.111288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660bf0c-d14d-4d1f-90b0-b81465bd83be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:43.111395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032e821-8037-4c94-aac4-2c2bbbdc53d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:43.111951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660bf0c-d14d-4d1f-90b0-b81465bd83be None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:43.112235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660bf0c-d14d-4d1f-90b0-b81465bd83be None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:43.112513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660bf0c-d14d-4d1f-90b0-b81465bd83be None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:43.112786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032e821-8037-4c94-aac4-2c2bbbdc53d3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:43.113268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660bf0c-d14d-4d1f-90b0-b81465bd83be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:43.113576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660bf0c-d14d-4d1f-90b0-b81465bd83be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:43.114950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660bf0c-d14d-4d1f-90b0-b81465bd83be None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:43.120381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/role_assignments?user.id=c06586f3b93746d589ca131fe0aad76b&scope.domain.id=default&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:04:43.133802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:04:45.362662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d699-2c4a-420b-8ead-8f42a06185e0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:45.363170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d699-2c4a-420b-8ead-8f42a06185e0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:45.363170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d699-2c4a-420b-8ead-8f42a06185e0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:45.364704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:45.370493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caee388-243d-4a62-8b44-88867366a01b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:45.370906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caee388-243d-4a62-8b44-88867366a01b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:45.371103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caee388-243d-4a62-8b44-88867366a01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:45.393831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8caee388-243d-4a62-8b44-88867366a01b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:45.395043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8caee388-243d-4a62-8b44-88867366a01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:45.484338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:45.514346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9d699-2c4a-420b-8ead-8f42a06185e0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:45.665344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9d699-2c4a-420b-8ead-8f42a06185e0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:45.707648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:45.713478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:45.713970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:45.714444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:45.725040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:04:45.728287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0b58f-9c6f-4b93-89fc-a46a3baf994b None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:45.731039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0b58f-9c6f-4b93-89fc-a46a3baf994b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:45.731039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0b58f-9c6f-4b93-89fc-a46a3baf994b None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:45.731039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:04:45.735890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5a02-04e8-4c6e-86c6-815e7f21ac00 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:45.736180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5a02-04e8-4c6e-86c6-815e7f21ac00 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:45.736463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5a02-04e8-4c6e-86c6-815e7f21ac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:04:45.740970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:45.741809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:45.742555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:04:45.745045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 13/25] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:45.755676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:45.763914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-05dc5a02-04e8-4c6e-86c6-815e7f21ac00 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:45.765289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-05dc5a02-04e8-4c6e-86c6-815e7f21ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:45.769132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:45.915258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d489f-7e50-45b6-a525-42e48c5aa12f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:45.923375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:45.982580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:45.983527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:45.983997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:45.984380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 13/26] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:04:45.984580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:45.985609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:45.986062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:45.987587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:45.996865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 14/27] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:04:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:04:46.005668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5a02-04e8-4c6e-86c6-815e7f21ac00 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:46.024462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:46.148998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:46.155895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5a02-04e8-4c6e-86c6-815e7f21ac00 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:46.192965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:46.194193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.194620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.195078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] PATH_INFO: `/v3/users/c06586f3b93746d589ca131fe0aad76b` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.196832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] RBAC: Authorizing `identity:get_user(user_id=c06586f3b93746d589ca131fe0aad76b)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:46.199127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:46.201064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 14/28] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:04:46 2022] GET /identity/v3/users/c06586f3b93746d589ca131fe0aad76b => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:46.209478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:46.211421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba08e3-4982-470d-86e8-da9ec53f0b02 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:46.273725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df392767-471f-4f0c-ac48-e2234486584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:46.274768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392767-471f-4f0c-ac48-e2234486584d None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.275292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392767-471f-4f0c-ac48-e2234486584d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.275767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392767-471f-4f0c-ac48-e2234486584d None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.276737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df392767-471f-4f0c-ac48-e2234486584d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:46.278398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df392767-471f-4f0c-ac48-e2234486584d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:46.285015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:46.355929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826bd78-99f9-47fd-a3ff-2e4898bdc68e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:46.392579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677e467-7fd9-41d6-ae01-423a733d4ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:46.394856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e467-7fd9-41d6-ae01-423a733d4ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.395613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e467-7fd9-41d6-ae01-423a733d4ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.396384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e467-7fd9-41d6-ae01-423a733d4ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.397993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677e467-7fd9-41d6-ae01-423a733d4ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:46.399919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677e467-7fd9-41d6-ae01-423a733d4ba1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:46.401972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 15/29] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:46.403446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046337-8cba-434d-9728-06bebe5b599b None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.403937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046337-8cba-434d-9728-06bebe5b599b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.404493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046337-8cba-434d-9728-06bebe5b599b None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.406009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 16/30] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:46.407322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca991eb-84f9-4937-be01-e4d2f020376c None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.407825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca991eb-84f9-4937-be01-e4d2f020376c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.408356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca991eb-84f9-4937-be01-e4d2f020376c None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.409778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 17/31] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:46.410659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca991eb-84f9-4937-be01-e4d2f020376c None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:46.452322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 15/32] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Jun 24 09:04:46 2022] POST /identity/v3/roles => generated 219 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:04:46.454009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7229-8e42-4a05-8d50-d19292bdf527 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.454309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7229-8e42-4a05-8d50-d19292bdf527 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.454705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7229-8e42-4a05-8d50-d19292bdf527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.482327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-28fd7229-8e42-4a05-8d50-d19292bdf527 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:46.484431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-28fd7229-8e42-4a05-8d50-d19292bdf527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:46.550559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca991eb-84f9-4937-be01-e4d2f020376c None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:46.584834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:46.630005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 16/33] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:46.631506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b7c47-5fe7-4e0e-af9b-29e91d41e4bb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.631617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:46.631794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b7c47-5fe7-4e0e-af9b-29e91d41e4bb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.632079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b7c47-5fe7-4e0e-af9b-29e91d41e4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.632708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.633064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.633515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] PATH_INFO: `/v3/domains/default/users/c06586f3b93746d589ca131fe0aad76b/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.635867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c06586f3b93746d589ca131fe0aad76b, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:46.637676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:46.667625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-761b7c47-5fe7-4e0e-af9b-29e91d41e4bb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:46.669258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-761b7c47-5fe7-4e0e-af9b-29e91d41e4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:46.731257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:46.754467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 17/34] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:46.755624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b7c47-5fe7-4e0e-af9b-29e91d41e4bb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:46.817865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 18/35] 10.222.0.52 () {58 vars in 1361 bytes} [Fri Jun 24 09:04:46 2022] PUT /identity/v3/domains/default/users/c06586f3b93746d589ca131fe0aad76b/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:04:46.819230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:46.898805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b7c47-5fe7-4e0e-af9b-29e91d41e4bb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:46.957345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fcca1-cde0-4569-a11b-8ac63c9ad8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:46.958467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fcca1-cde0-4569-a11b-8ac63c9ad8d2 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.958763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fcca1-cde0-4569-a11b-8ac63c9ad8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.959041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fcca1-cde0-4569-a11b-8ac63c9ad8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.960119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fcca1-cde0-4569-a11b-8ac63c9ad8d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:46.961998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fcca1-cde0-4569-a11b-8ac63c9ad8d2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:46.988896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 18/36] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Jun 24 09:04:46 2022] POST /identity/v3/roles => generated 225 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:04:46.991718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e00ab-3a9c-492f-9950-4ec4f0bd5355 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:46.992833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e00ab-3a9c-492f-9950-4ec4f0bd5355 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:46.994921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e00ab-3a9c-492f-9950-4ec4f0bd5355 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:46.998459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2f7f9-cfab-4fe4-b545-a130688a8b9e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:47.000950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 19/37] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:47.008025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adf69d-7c59-4656-ae8e-fc3109a6d764 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.008759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adf69d-7c59-4656-ae8e-fc3109a6d764 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.009333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adf69d-7c59-4656-ae8e-fc3109a6d764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.040048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e0adf69d-7c59-4656-ae8e-fc3109a6d764 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:47.041656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e0adf69d-7c59-4656-ae8e-fc3109a6d764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:47.095273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:47.096111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.096396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.096697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.105001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:47.128294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 20/38] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:04:47.133745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:47.136345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:04:47.155598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:47.156478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:47.157400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:04:47.157400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:04:47.157400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:04:47.157400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:04:47.159845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 19/39] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:47.165569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adf69d-7c59-4656-ae8e-fc3109a6d764 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:47.185218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:47.222730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:47.307587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adf69d-7c59-4656-ae8e-fc3109a6d764 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:47.316517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf594af-99a1-471a-be6f-7627070ee629 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:47.343868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19e9d9-6b44-4355-afef-babfc6c68ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:47.344793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e9d9-6b44-4355-afef-babfc6c68ebe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.345302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e9d9-6b44-4355-afef-babfc6c68ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.345556 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e9d9-6b44-4355-afef-babfc6c68ebe None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.346255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19e9d9-6b44-4355-afef-babfc6c68ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:47.346564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19e9d9-6b44-4355-afef-babfc6c68ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:47.347944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19e9d9-6b44-4355-afef-babfc6c68ebe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:47.355304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca36d7-f92e-47c7-aee4-5de826805126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:47.355977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca36d7-f92e-47c7-aee4-5de826805126 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.356263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca36d7-f92e-47c7-aee4-5de826805126 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.356542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca36d7-f92e-47c7-aee4-5de826805126 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.357322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca36d7-f92e-47c7-aee4-5de826805126 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:47.358709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca36d7-f92e-47c7-aee4-5de826805126 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:47.367497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 21/40] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:47.373955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e867b37-9225-4066-8f20-35c2c5a5a028 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.374252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e867b37-9225-4066-8f20-35c2c5a5a028 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.374533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e867b37-9225-4066-8f20-35c2c5a5a028 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.375990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 22/41] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:47.376524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e867b37-9225-4066-8f20-35c2c5a5a028 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:47.382935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 20/42] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Jun 24 09:04:47 2022] POST /identity/v3/roles => generated 223 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:04:47.382935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b524f-17c3-44ee-bb01-de33d75d7f23 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.383211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b524f-17c3-44ee-bb01-de33d75d7f23 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.383498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b524f-17c3-44ee-bb01-de33d75d7f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.403687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-845b524f-17c3-44ee-bb01-de33d75d7f23 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:47.405763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-845b524f-17c3-44ee-bb01-de33d75d7f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:47.425989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:47.499556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 21/43] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:47.523029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b524f-17c3-44ee-bb01-de33d75d7f23 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:47.540055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:47.625154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e867b37-9225-4066-8f20-35c2c5a5a028 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:47.651591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b524f-17c3-44ee-bb01-de33d75d7f23 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:47.676533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac30a04-fd94-4d18-8b20-bfc9c67bb543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:47.677326 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30a04-fd94-4d18-8b20-bfc9c67bb543 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.677536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30a04-fd94-4d18-8b20-bfc9c67bb543 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.677814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30a04-fd94-4d18-8b20-bfc9c67bb543 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.678528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac30a04-fd94-4d18-8b20-bfc9c67bb543 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:47.679969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac30a04-fd94-4d18-8b20-bfc9c67bb543 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:47.689247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:47.690057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.690341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.690620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.691800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:47.693281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:47.694903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 22/44] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:04:47.701276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:47.709814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 23/45] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:04:47 2022] POST /identity/v3/projects => generated 301 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:04:47.803373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:47.825870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db9c1b-cf4a-454f-a695-752cedcbaf3b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:47.866897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543a768-ccb4-41e0-bd72-7e6671e59010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:47.867587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543a768-ccb4-41e0-bd72-7e6671e59010 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:47.867874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543a768-ccb4-41e0-bd72-7e6671e59010 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:47.868204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543a768-ccb4-41e0-bd72-7e6671e59010 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:47.868927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543a768-ccb4-41e0-bd72-7e6671e59010 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:47.871972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543a768-ccb4-41e0-bd72-7e6671e59010 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:47.904653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:04:47 2022] POST /identity/v3/projects => generated 312 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:04:48.994674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abe158-b9b8-45cb-a61f-48fad98b9419 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:48.995186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abe158-b9b8-45cb-a61f-48fad98b9419 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:48.995267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abe158-b9b8-45cb-a61f-48fad98b9419 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:48.996641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 24/47] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:04:49.001840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffda03-b6a8-4373-b352-5adbe8be8bf3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.002211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffda03-b6a8-4373-b352-5adbe8be8bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.002409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffda03-b6a8-4373-b352-5adbe8be8bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:49.019833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-50ffda03-b6a8-4373-b352-5adbe8be8bf3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:49.021179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-50ffda03-b6a8-4373-b352-5adbe8be8bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:49.088444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:49.111665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abe158-b9b8-45cb-a61f-48fad98b9419 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:49.199440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:49.213638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abe158-b9b8-45cb-a61f-48fad98b9419 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:49.246214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:49.246940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.247227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.247508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:49.257766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:04:49.265422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:49.266425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:04:49.267345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 25/49] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:49.273908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffda03-b6a8-4373-b352-5adbe8be8bf3 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:49.335741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:49.376115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffda03-b6a8-4373-b352-5adbe8be8bf3 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:49.409387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:49.410057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.410343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.410619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:49.411365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:49.411674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:49.413092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:49.425237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 25/50] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:49.430907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:49.529444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a69aa-dfc4-4f45-8d27-4580ea926a85 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:49.561989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:49.562857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.563140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.563415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] PATH_INFO: `/v3/users/c06586f3b93746d589ca131fe0aad76b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:49.564631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] RBAC: Authorizing `identity:get_user(user_id=c06586f3b93746d589ca131fe0aad76b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:49.566085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:49.567871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:04:49 2022] GET /identity/v3/users/c06586f3b93746d589ca131fe0aad76b => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:49.573711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:49.673161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dc9e2-520c-4c82-8aef-52cc6eb0e8aa None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:49.802688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:49.812189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694af888-6655-4a74-968c-b942fba97778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:49.813018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af888-6655-4a74-968c-b942fba97778 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.813305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af888-6655-4a74-968c-b942fba97778 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.813586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af888-6655-4a74-968c-b942fba97778 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:49.814768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694af888-6655-4a74-968c-b942fba97778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:49.816216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694af888-6655-4a74-968c-b942fba97778 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:49.817882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 26/52] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:49.824023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:49.843195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:49.927665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707f94b-7d7e-4672-8b50-f90a76ae5b0a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:49.962012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f97c34-ccdc-4b08-8971-cc406ade45d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:49.962667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f97c34-ccdc-4b08-8971-cc406ade45d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.962975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f97c34-ccdc-4b08-8971-cc406ade45d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.963252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f97c34-ccdc-4b08-8971-cc406ade45d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:49.964073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f97c34-ccdc-4b08-8971-cc406ade45d0 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.domain.id=default, user.id=c06586f3b93746d589ca131fe0aad76b) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:49.964382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f97c34-ccdc-4b08-8971-cc406ade45d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:49.965862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f97c34-ccdc-4b08-8971-cc406ade45d0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:49.971795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1322 bytes} [Fri Jun 24 09:04:49 2022] GET /identity/v3/role_assignments?user.id=c06586f3b93746d589ca131fe0aad76b&scope.domain.id=default&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:49.976483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bfb47-2306-4c28-8bc0-597c3320bc53 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.976799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bfb47-2306-4c28-8bc0-597c3320bc53 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.977083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bfb47-2306-4c28-8bc0-597c3320bc53 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:49.978490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 27/54] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:49.984740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5ba5c-8271-4405-b5cf-703bf4cd39ab None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:49.985038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5ba5c-8271-4405-b5cf-703bf4cd39ab None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:49.985325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5ba5c-8271-4405-b5cf-703bf4cd39ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:50.003823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-07b5ba5c-8271-4405-b5cf-703bf4cd39ab None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:50.005081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-07b5ba5c-8271-4405-b5cf-703bf4cd39ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:50.078877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 28/55] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:50.102430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bfb47-2306-4c28-8bc0-597c3320bc53 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:50.204964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bfb47-2306-4c28-8bc0-597c3320bc53 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:50.236705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d954ec1-0f6a-40d9-8e8f-7354e1ac16d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:50.237502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d954ec1-0f6a-40d9-8e8f-7354e1ac16d7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:50.237787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d954ec1-0f6a-40d9-8e8f-7354e1ac16d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:50.238065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d954ec1-0f6a-40d9-8e8f-7354e1ac16d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:50.239226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d954ec1-0f6a-40d9-8e8f-7354e1ac16d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:50.240696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d954ec1-0f6a-40d9-8e8f-7354e1ac16d7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:50.242302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 28/56] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:50.248612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5ba5c-8271-4405-b5cf-703bf4cd39ab None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:50.260573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:50.356698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5ba5c-8271-4405-b5cf-703bf4cd39ab None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:50.389626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60835f9b-8105-46aa-b7da-aa43ad824823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:50.390564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60835f9b-8105-46aa-b7da-aa43ad824823 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:50.390859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60835f9b-8105-46aa-b7da-aa43ad824823 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:50.391142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60835f9b-8105-46aa-b7da-aa43ad824823 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:50.391867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60835f9b-8105-46aa-b7da-aa43ad824823 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:50.393451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60835f9b-8105-46aa-b7da-aa43ad824823 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:50.418363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 29/57] 10.222.0.52 () {60 vars in 1140 bytes} [Fri Jun 24 09:04:50 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:04:52.444958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ec0dd-3196-4e44-a207-73db8f156d86 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:52.445444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ec0dd-3196-4e44-a207-73db8f156d86 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:52.445444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ec0dd-3196-4e44-a207-73db8f156d86 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:52.446818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 29/58] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:52.452531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f952f-b4b9-490c-bc6c-f8106ca9c362 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:52.452954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f952f-b4b9-490c-bc6c-f8106ca9c362 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:52.453115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f952f-b4b9-490c-bc6c-f8106ca9c362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:52.469285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a33f952f-b4b9-490c-bc6c-f8106ca9c362 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:52.470468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a33f952f-b4b9-490c-bc6c-f8106ca9c362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:52.525083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:52.536152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 30/59] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:04:52.561834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ec0dd-3196-4e44-a207-73db8f156d86 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:52.627849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:52.672406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ec0dd-3196-4e44-a207-73db8f156d86 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:52.705245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28038d86-75a2-4755-9ef4-ca3cc909eefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:52.705978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28038d86-75a2-4755-9ef4-ca3cc909eefb None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:52.706264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28038d86-75a2-4755-9ef4-ca3cc909eefb None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:52.706544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28038d86-75a2-4755-9ef4-ca3cc909eefb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:52.707702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28038d86-75a2-4755-9ef4-ca3cc909eefb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:52.709182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28038d86-75a2-4755-9ef4-ca3cc909eefb None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:52.710796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 30/60] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:52.716312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f952f-b4b9-490c-bc6c-f8106ca9c362 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:52.816470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f952f-b4b9-490c-bc6c-f8106ca9c362 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:52.850062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b803d-c8d8-4d83-9163-456c1d8f656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:52.850725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b803d-c8d8-4d83-9163-456c1d8f656d None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:52.851006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b803d-c8d8-4d83-9163-456c1d8f656d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:52.851282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b803d-c8d8-4d83-9163-456c1d8f656d None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:52.851995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b803d-c8d8-4d83-9163-456c1d8f656d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:52.853978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b803d-c8d8-4d83-9163-456c1d8f656d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:52.878111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 31/61] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:04:52 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:04:56.018009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1751a9-7ced-4bf9-a4df-fac266ec9e68 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.023620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1751a9-7ced-4bf9-a4df-fac266ec9e68 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.024138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1751a9-7ced-4bf9-a4df-fac266ec9e68 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.025839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 31/62] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:56.040357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19237e4-0e5a-4e4d-9a28-eff08001e2f7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.041212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19237e4-0e5a-4e4d-9a28-eff08001e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.041724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19237e4-0e5a-4e4d-9a28-eff08001e2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.065653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e19237e4-0e5a-4e4d-9a28-eff08001e2f7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:56.066884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e19237e4-0e5a-4e4d-9a28-eff08001e2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:56.126097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:56.157955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 32/63] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:56.195316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1751a9-7ced-4bf9-a4df-fac266ec9e68 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:56.218185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd8216-9fe1-41a6-b950-0e5f907133e4 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.218479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd8216-9fe1-41a6-b950-0e5f907133e4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.218758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd8216-9fe1-41a6-b950-0e5f907133e4 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.220291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 33/64] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:56.234250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e407c-28e0-42f8-af8e-63929541e156 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.234538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e407c-28e0-42f8-af8e-63929541e156 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.234816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e407c-28e0-42f8-af8e-63929541e156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.261310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-038e407c-28e0-42f8-af8e-63929541e156 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:56.262506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-038e407c-28e0-42f8-af8e-63929541e156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:56.376065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1751a9-7ced-4bf9-a4df-fac266ec9e68 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:56.384049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 34/65] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:04:56.385703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545e1b-5d26-4f16-b578-8099b5ee4f0b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.386458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545e1b-5d26-4f16-b578-8099b5ee4f0b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.386888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545e1b-5d26-4f16-b578-8099b5ee4f0b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.388780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 35/66] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:56.393900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49672a-fed1-4d88-95c0-b881de2ae725 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.394539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49672a-fed1-4d88-95c0-b881de2ae725 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.395408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49672a-fed1-4d88-95c0-b881de2ae725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.420333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7d49672a-fed1-4d88-95c0-b881de2ae725 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:56.421905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7d49672a-fed1-4d88-95c0-b881de2ae725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:56.424690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:56.430969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:56.433386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.433746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.434091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.452146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:04:56.461219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:56.462072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:04:56.462817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:04:56.465842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 32/67] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:56.466819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:56.506360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 36/68] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:56.519605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639373-7bac-42a9-a32e-870b8629424c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.519605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639373-7bac-42a9-a32e-870b8629424c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.519605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639373-7bac-42a9-a32e-870b8629424c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.519605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 37/69] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:56.519605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e639373-7bac-42a9-a32e-870b8629424c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:56.562903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:56.591851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff1199-20ba-488f-883a-082878f4b639 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:56.632780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db087abc-4766-4bcd-b874-bee5ede8d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:56.633841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087abc-4766-4bcd-b874-bee5ede8d3ca None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.634128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087abc-4766-4bcd-b874-bee5ede8d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.634307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e639373-7bac-42a9-a32e-870b8629424c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:56.634418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087abc-4766-4bcd-b874-bee5ede8d3ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.650002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db087abc-4766-4bcd-b874-bee5ede8d3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-db087abc-4766-4bcd-b874-bee5ede8d3ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:04:56.658214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:56.659108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:04:56.659853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 33/70] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:04:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:56.661154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37bab-21d6-42cc-8c95-9cba323e9e5e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.661154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37bab-21d6-42cc-8c95-9cba323e9e5e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.661154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37bab-21d6-42cc-8c95-9cba323e9e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.676807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:56.677483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.677823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.678105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.678260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-95c37bab-21d6-42cc-8c95-9cba323e9e5e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:56.678855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:56.679191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:56.679429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-95c37bab-21d6-42cc-8c95-9cba323e9e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:56.680566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:56.691038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 38/71] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:04:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:56.691852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:56.736038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:56.758059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 34/72] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:56.758960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c37bab-21d6-42cc-8c95-9cba323e9e5e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:56.800704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:56.803777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39ceb-956b-418e-876a-a6ac5333a6e6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:56.838608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:56.839494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.839784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.840064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.850056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:04:56.857635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:56.858487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:04:56.859282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 39/73] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:56.859902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:56.870564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c37bab-21d6-42cc-8c95-9cba323e9e5e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:56.905730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bcf98-3789-4167-a018-a88b448d2ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:56.906329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bcf98-3789-4167-a018-a88b448d2ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.906616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bcf98-3789-4167-a018-a88b448d2ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.906894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bcf98-3789-4167-a018-a88b448d2ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.907723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bcf98-3789-4167-a018-a88b448d2ad0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:56.908034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bcf98-3789-4167-a018-a88b448d2ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:56.909461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bcf98-3789-4167-a018-a88b448d2ad0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:56.917860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 35/74] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:04:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:56.919221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907978d4-3d45-42ca-9ea0-487db9867ba1 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:56.919515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907978d4-3d45-42ca-9ea0-487db9867ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:56.919796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907978d4-3d45-42ca-9ea0-487db9867ba1 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:56.921204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 36/75] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:04:56.922098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907978d4-3d45-42ca-9ea0-487db9867ba1 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:56.965835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:56.970007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a70fe-c382-4ac1-927e-836c85b4e8d0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.003254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.004136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.004425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.004740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.018380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.019833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.021866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 40/76] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:04:56 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.022656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.032720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907978d4-3d45-42ca-9ea0-487db9867ba1 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.042151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.067399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.068203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.068492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.068807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.079407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:04:57.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:04:57.087903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 37/77] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:04:57.089821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.124837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4715b-129a-4071-8ca2-6d823864317a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.160904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563ba2e-b47b-46b7-99f6-516027ae3141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.161523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563ba2e-b47b-46b7-99f6-516027ae3141 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.161811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563ba2e-b47b-46b7-99f6-516027ae3141 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.162092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563ba2e-b47b-46b7-99f6-516027ae3141 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.162847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563ba2e-b47b-46b7-99f6-516027ae3141 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:57.163159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563ba2e-b47b-46b7-99f6-516027ae3141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.164570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563ba2e-b47b-46b7-99f6-516027ae3141 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.172895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 41/78] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.174091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb651-4930-49fb-b177-d41855786c44 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.174386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb651-4930-49fb-b177-d41855786c44 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.174670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcb651-4930-49fb-b177-d41855786c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.189142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17729f34-a4c7-47b9-9c2b-2ef81f32907e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.190522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f9fcb651-4930-49fb-b177-d41855786c44 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:04:57.190723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.191735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f9fcb651-4930-49fb-b177-d41855786c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:57.225171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.226047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.226334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.226608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.227817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.229308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.230935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 38/79] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.231694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.262789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 42/80] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:04:57.263872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb651-4930-49fb-b177-d41855786c44 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.278654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.329781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce44433-0af3-4af3-a285-30d710eda640 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.363677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.364321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.364643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.364920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.369474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcb651-4930-49fb-b177-d41855786c44 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.403462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.403980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.404092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.404179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.404457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.405633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:57.405967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.407358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.415955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 43/81] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.416993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.433594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.435067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.488693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.521033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49202-1145-46a5-b404-5bda7bb10e51 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.536752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 39/82] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:04:57.537613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.558787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.559663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.559948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.560223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] PATH_INFO: `/v3/users/c06586f3b93746d589ca131fe0aad76b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.561444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] RBAC: Authorizing `identity:get_user(user_id=c06586f3b93746d589ca131fe0aad76b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.562890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.564672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 44/83] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/users/c06586f3b93746d589ca131fe0aad76b => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.565428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.610497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.640186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f954b0-834d-4a22-885b-107da21f3181 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.663341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fa842-12ca-4b6c-a0c2-9b773552d308 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.679650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.680520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.680835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.681116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.699589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.700380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.700704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.700989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.702161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.703604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 45/84] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.706004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.717849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.749572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.751061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.807964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1cb59-778c-4483-bc37-7bb58cd549f5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.818627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.841651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.842533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.842826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.843107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.844308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.845791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.847422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 46/85] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.848185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.851296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 40/86] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:04:57.852129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:57.926306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:57.950348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2c54-a114-4f47-b26e-6a2db01af3c8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.951417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31011a04-5afb-4a70-8994-fd9f7bfee223 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:57.987509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.987755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:57.988159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.988444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.988647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:57.988737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.988940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:57.989223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:57.990406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:57.992667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:57.993852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 47/87] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:57.994606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.030782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:58.060171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.061702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.097934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9245ef-ea59-4ea0-8822-12cee36c276b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.131653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.132329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.132651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.132933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.133647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.135497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.151917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:58.163405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 48/88] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:04:57 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:04:58.163405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.169038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 41/89] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:04:58.170013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.259610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:58.297206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633ab3-fc8f-4902-9238-981f21ea9d2d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.297444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d278a9-c742-4670-a29d-1d89c71a95c4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.340005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.340849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.341367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.341658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.341756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.341936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.342029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.342259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.343536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.345051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.346992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 42/90] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:58.347673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.430789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:58.439400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.440938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.472504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb484e-9743-41b6-9222-52328dcb3574 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.515552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.516145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.516433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.516748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.517602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=c777cddf95b3464587cd81664f9af1ee) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:58.517920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.519781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.527852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 43/91] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/role_assignments?user.id=c777cddf95b3464587cd81664f9af1ee&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:58.529108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.531619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:58.561481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 49/92] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:04:58.562152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.662366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86324b2-024e-493f-9103-4eec6d180e0e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.688229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b63aca-7622-46eb-9045-c6b748c692a4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.692956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:58.702308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.703538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.704105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.704602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.706127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.707819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.709848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 44/93] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:58.710874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.740701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.741676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.741787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.742021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.742877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] RBAC: Adding query filter params (role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=c777cddf95b3464587cd81664f9af1ee) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:58.743192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.744684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.755220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 50/94] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/role_assignments?user.id=c777cddf95b3464587cd81664f9af1ee&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=cc730cc298404131a99c3547377a5e49 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:58.756175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.799637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:58.835718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a039b6-18b4-4d7f-8182-1dcdf6849ace None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.874505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.875771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.876270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.876824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] PATH_INFO: `/v3/projects/865cfaca93154c65a11411d92cd85b24` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.878444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] RBAC: Authorizing `identity:get_project(project_id=865cfaca93154c65a11411d92cd85b24)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.880176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.889079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 45/95] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/projects/865cfaca93154c65a11411d92cd85b24 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:58.902044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:58.941940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284643e2-78c7-4486-a12f-79e00b98902f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:58.981204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0df8c-3f0f-4b22-94db-ba50c82cbc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:58.981925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0df8c-3f0f-4b22-94db-ba50c82cbc7a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:58.982158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0df8c-3f0f-4b22-94db-ba50c82cbc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:58.982448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0df8c-3f0f-4b22-94db-ba50c82cbc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:58.983284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0df8c-3f0f-4b22-94db-ba50c82cbc7a None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=c06586f3b93746d589ca131fe0aad76b) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:58.983598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0df8c-3f0f-4b22-94db-ba50c82cbc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:58.986224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0df8c-3f0f-4b22-94db-ba50c82cbc7a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:58.992470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 51/96] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/role_assignments?user.id=c06586f3b93746d589ca131fe0aad76b&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:04:59.044289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1136701-6321-4e36-828c-055bacf01165 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:59.085559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bc49c-5a67-463d-9f6b-c8dc801d43c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:59.087012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bc49c-5a67-463d-9f6b-c8dc801d43c6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:59.087520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bc49c-5a67-463d-9f6b-c8dc801d43c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:59.088016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bc49c-5a67-463d-9f6b-c8dc801d43c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:59.089664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bc49c-5a67-463d-9f6b-c8dc801d43c6 None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=865cfaca93154c65a11411d92cd85b24, user.id=c777cddf95b3464587cd81664f9af1ee) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:04:59.090190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bc49c-5a67-463d-9f6b-c8dc801d43c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:59.091880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bc49c-5a67-463d-9f6b-c8dc801d43c6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:59.102766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 46/97] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:04:58 2022] GET /identity/v3/role_assignments?user.id=c777cddf95b3464587cd81664f9af1ee&scope.project.id=865cfaca93154c65a11411d92cd85b24&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:00.714204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be57af-be51-408a-b47e-aa5919c16530 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:00.714714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be57af-be51-408a-b47e-aa5919c16530 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:00.714787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be57af-be51-408a-b47e-aa5919c16530 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:00.716120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 52/98] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:00.721472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf258-e42c-41d6-a1e0-be5503bf27d3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:00.722279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf258-e42c-41d6-a1e0-be5503bf27d3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:00.722752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf258-e42c-41d6-a1e0-be5503bf27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:00.728005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:00.740809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-663cf258-e42c-41d6-a1e0-be5503bf27d3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:00.742278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-663cf258-e42c-41d6-a1e0-be5503bf27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:00.813050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 47/99] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:00.851192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be57af-be51-408a-b47e-aa5919c16530 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:00.885058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:00.984400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be57af-be51-408a-b47e-aa5919c16530 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:01.030248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:01.031054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.031339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.031611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.049041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:01.060793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:01.061678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:01.062443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 53/100] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:01.067739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf258-e42c-41d6-a1e0-be5503bf27d3 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:01.195238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf258-e42c-41d6-a1e0-be5503bf27d3 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:01.236715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:01.246697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3643d3b-2d2d-4a19-b562-c73d818d4e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:01.247648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3643d3b-2d2d-4a19-b562-c73d818d4e81 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.248133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3643d3b-2d2d-4a19-b562-c73d818d4e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.249414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3643d3b-2d2d-4a19-b562-c73d818d4e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.250377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3643d3b-2d2d-4a19-b562-c73d818d4e81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:01.250903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3643d3b-2d2d-4a19-b562-c73d818d4e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:01.252578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3643d3b-2d2d-4a19-b562-c73d818d4e81 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:01.264636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 48/101] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:01.270333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:01.361088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e318e8-33f6-41ab-9d0f-1723c3958cce None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.361428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e318e8-33f6-41ab-9d0f-1723c3958cce None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.361661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e318e8-33f6-41ab-9d0f-1723c3958cce None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.363227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 49/102] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:01.370772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749f3b-09ef-4815-87b1-299ba9115989 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.371331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749f3b-09ef-4815-87b1-299ba9115989 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.371821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749f3b-09ef-4815-87b1-299ba9115989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.392646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:01.396177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-33749f3b-09ef-4815-87b1-299ba9115989 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:01.397672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-33749f3b-09ef-4815-87b1-299ba9115989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:01.414486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c51dd-d725-45dc-995c-75d0a0cd1d15 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:01.460166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:01.461012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.461304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.461586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] PATH_INFO: `/v3/users/691dc670ed5d4d018bd1b4fd160b8fc7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.481295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 50/103] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:01.488854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] RBAC: Authorizing `identity:get_user(user_id=691dc670ed5d4d018bd1b4fd160b8fc7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:01.488854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:01.492972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 54/104] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:01 2022] GET /identity/v3/users/691dc670ed5d4d018bd1b4fd160b8fc7 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:01.498633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749f3b-09ef-4815-87b1-299ba9115989 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:01.549634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:01.612739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749f3b-09ef-4815-87b1-299ba9115989 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:01.638289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:01.651328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:01.652307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.653898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.654402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.747969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabc604-a40e-4c08-b933-56b01d421465 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:01.789822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:01.793611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:01.813486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f3a0b-d103-4470-a944-33b48a9ebdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:01.814519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3a0b-d103-4470-a944-33b48a9ebdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.814977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3a0b-d103-4470-a944-33b48a9ebdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.815477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3a0b-d103-4470-a944-33b48a9ebdd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.816923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f3a0b-d103-4470-a944-33b48a9ebdd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:01.818538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f3a0b-d103-4470-a944-33b48a9ebdd3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:01.820746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 55/105] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:01.822116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8c6f6-6e74-4a8b-82b3-44af32ade33d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.822531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8c6f6-6e74-4a8b-82b3-44af32ade33d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.822969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8c6f6-6e74-4a8b-82b3-44af32ade33d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.824340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 56/106] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:01.830027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7f29-482e-4a38-a0ed-61fd9eef3ed2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:01.830513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7f29-482e-4a38-a0ed-61fd9eef3ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:01.831037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7f29-482e-4a38-a0ed-61fd9eef3ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:01.854146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9ccd7f29-482e-4a38-a0ed-61fd9eef3ed2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:01.856404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9ccd7f29-482e-4a38-a0ed-61fd9eef3ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:01.876812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:01.911437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 51/107] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:05:01.912681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:01.939164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:02.035973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 57/108] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:02.036333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6392781-e79b-4675-9f79-8d2a0eb92b82 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.036893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd7f29-482e-4a38-a0ed-61fd9eef3ed2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:02.096897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:02.097962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.099607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.100042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.101817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:02.103296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:02.156919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 52/109] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:05:01 2022] POST /identity/v3/projects => generated 302 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:02.158126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:02.217343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd7f29-482e-4a38-a0ed-61fd9eef3ed2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.263310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:02.264333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326242a-f297-478a-80d3-432c6bdbf7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:02.265492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326242a-f297-478a-80d3-432c6bdbf7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.265959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326242a-f297-478a-80d3-432c6bdbf7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.266481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326242a-f297-478a-80d3-432c6bdbf7e8 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.268444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326242a-f297-478a-80d3-432c6bdbf7e8 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:02.270639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326242a-f297-478a-80d3-432c6bdbf7e8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:02.273654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 58/110] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:02 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:02.275324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01747e-30af-47ff-affd-77c320035820 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.275889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01747e-30af-47ff-affd-77c320035820 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.277169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01747e-30af-47ff-affd-77c320035820 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.278851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 59/111] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:02.280243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fd927-3742-4fd2-bf85-d45180f9c096 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.280773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fd927-3742-4fd2-bf85-d45180f9c096 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.281241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fd927-3742-4fd2-bf85-d45180f9c096 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.282652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 60/112] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:02.288149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fd927-3742-4fd2-bf85-d45180f9c096 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:02.294660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd2d83-9755-4c09-823b-c244e50ae31e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.305345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:02.353111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24f777-72e4-40d1-b1f9-531755a85053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:02.354582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f777-72e4-40d1-b1f9-531755a85053 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.354582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f777-72e4-40d1-b1f9-531755a85053 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.354582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f777-72e4-40d1-b1f9-531755a85053 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.368807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24f777-72e4-40d1-b1f9-531755a85053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-9b24f777-72e4-40d1-b1f9-531755a85053 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:02.378146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:02.379197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:02.380123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 53/113] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:02.385000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3514f7-698b-4073-9e84-12dc5b9af6bd None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.385300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3514f7-698b-4073-9e84-12dc5b9af6bd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.386808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3514f7-698b-4073-9e84-12dc5b9af6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.406797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7d3514f7-698b-4073-9e84-12dc5b9af6bd None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:02.407664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7d3514f7-698b-4073-9e84-12dc5b9af6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:02.424391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fd927-3742-4fd2-bf85-d45180f9c096 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.462043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03702b2-4580-4e6d-829b-750c07e69324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:02.462951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03702b2-4580-4e6d-829b-750c07e69324 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.463058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03702b2-4580-4e6d-829b-750c07e69324 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.463264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03702b2-4580-4e6d-829b-750c07e69324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.464703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03702b2-4580-4e6d-829b-750c07e69324 None admin] RBAC: Adding query filter params (role.id=f5a4836824db455d937bdc03a76f1c76, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=691dc670ed5d4d018bd1b4fd160b8fc7) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:02.465022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03702b2-4580-4e6d-829b-750c07e69324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:02.466520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03702b2-4580-4e6d-829b-750c07e69324 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:02.481191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 61/114] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:02 2022] GET /identity/v3/role_assignments?user.id=691dc670ed5d4d018bd1b4fd160b8fc7&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=f5a4836824db455d937bdc03a76f1c76 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:02.483477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c42e-5d16-493f-9af9-427280960141 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.484113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c42e-5d16-493f-9af9-427280960141 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.484667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872c42e-5d16-493f-9af9-427280960141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.485833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 54/115] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:02.486490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3514f7-698b-4073-9e84-12dc5b9af6bd None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:02.508537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a872c42e-5d16-493f-9af9-427280960141 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:02.511377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a872c42e-5d16-493f-9af9-427280960141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:02.581535 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:02.597793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 62/116] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:02.608275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02209-e996-41b6-b165-799357a9db4d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.608484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02209-e996-41b6-b165-799357a9db4d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.609387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02209-e996-41b6-b165-799357a9db4d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.619337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 63/117] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:02.619761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02209-e996-41b6-b165-799357a9db4d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:02.634280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:02.644479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3514f7-698b-4073-9e84-12dc5b9af6bd None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.684089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0be8f1-e16f-4a2a-bae5-5bfe9bd3752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:02.685130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0be8f1-e16f-4a2a-bae5-5bfe9bd3752c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.685567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0be8f1-e16f-4a2a-bae5-5bfe9bd3752c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.685987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0be8f1-e16f-4a2a-bae5-5bfe9bd3752c None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.686885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0be8f1-e16f-4a2a-bae5-5bfe9bd3752c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:02.687328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0be8f1-e16f-4a2a-bae5-5bfe9bd3752c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:02.688913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0be8f1-e16f-4a2a-bae5-5bfe9bd3752c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:02.697497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 55/118] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:02.699628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e66e-3c3f-42c2-901a-224330e4c7ee None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.700068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e66e-3c3f-42c2-901a-224330e4c7ee None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.700499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e66e-3c3f-42c2-901a-224330e4c7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.717168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-30e8e66e-3c3f-42c2-901a-224330e4c7ee None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:02.718758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-30e8e66e-3c3f-42c2-901a-224330e4c7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:02.758153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02209-e996-41b6-b165-799357a9db4d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.789678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 56/119] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:02.790850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e66e-3c3f-42c2-901a-224330e4c7ee None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:02.813813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:02.814625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.814917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.815201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.837678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:02.846161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:02.847065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:02.847853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 64/120] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:02.848499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:02.863532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:02.927463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e66e-3c3f-42c2-901a-224330e4c7ee None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.936294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:02.969632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:02.970977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:02.971494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:02.972017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:02.976408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5584-3071-4869-b10f-88f07ad57bf4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:02.985115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:02.993778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:02.994614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:02.995989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:03.004532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 57/121] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:03.006051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.014433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.015326 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.015617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.015901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.017603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.019509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.023525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 65/122] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:02 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.023525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.129435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a1586-596e-4482-9bc1-999a49fb1785 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.132452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c5c0c-4116-4887-bbab-cdc066b0a263 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.147993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:03.167686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.168347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.168666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.168911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.169029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.169697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:03.170008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.170253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.170665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.171081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.171391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.172849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:03.180893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 66/123] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.185497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:03.187004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:03.195336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:03.196389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:03.197320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:03.199338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 58/124] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:03.200391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.290660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e424d8e-5847-4dce-91dd-287796546119 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.312884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651b42-0c86-49ec-8f77-dfd98b6ede81 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.326787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae66da-6514-4933-b646-411518692761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.327470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae66da-6514-4933-b646-411518692761 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.327765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae66da-6514-4933-b646-411518692761 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.328050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae66da-6514-4933-b646-411518692761 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.328955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae66da-6514-4933-b646-411518692761 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:03.329268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae66da-6514-4933-b646-411518692761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.330754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae66da-6514-4933-b646-411518692761 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.340321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 67/125] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.341235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae66da-6514-4933-b646-411518692761 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.351384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.352779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.353269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.353744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.356047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.357922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.359922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 59/126] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.361109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.406407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:03.415035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:03.453209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae66da-6514-4933-b646-411518692761 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.476022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e4c5c-3fa1-424e-875b-624c7b8b9fa3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.493040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.494384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.495209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.495656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.497041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.499178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.501677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 68/127] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.502654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.518343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.519255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.519608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.519959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.520913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:03.521429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.523217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.533167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 60/128] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.534092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.639024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4cebf-5450-468b-b5a9-a424d3b6f15c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.647046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da72c5-3956-4a85-a61f-b659c6433047 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.659655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:03.680384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:03.695040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.695953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.696247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.696533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.698402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.699299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.699582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.699861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/c777cddf95b3464587cd81664f9af1ee/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.701049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.702516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.704596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 69/129] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.705379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.706293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] RBAC: Authorizing `identity:create_grant(project_id=473c53bf5c8740e4aaabb9c9465bd165, user_id=c777cddf95b3464587cd81664f9af1ee, role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.708037 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.727396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 61/130] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:03 2022] PUT /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/c777cddf95b3464587cd81664f9af1ee/roles/cc730cc298404131a99c3547377a5e49 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:03.728907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.865532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576c85-6c6f-4db4-9db3-fd0c7099ad09 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.909914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.911478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.912224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.912673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] PATH_INFO: `/v3/users/c06586f3b93746d589ca131fe0aad76b` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.914046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be805ceb-d1e6-4c26-912a-67dab97ab22a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:03.914561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=c06586f3b93746d589ca131fe0aad76b)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.916347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.920169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 62/131] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/users/c06586f3b93746d589ca131fe0aad76b => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.921316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.954102 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:03.955166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:03.955432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:03.955786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:03.957082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:03.958617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:03.960600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 70/132] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:03.961273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:03.968367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:04.051789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:04.075111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7f5f7-23ea-4f8f-9520-135f51abf7f4 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:04.119413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096dcec7-35ab-424e-928b-f9e2dc201db4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:04.136261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:04.137225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.137586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.137845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] PATH_INFO: `/v3/projects/865cfaca93154c65a11411d92cd85b24` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.139061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] RBAC: Authorizing `identity:get_project(project_id=865cfaca93154c65a11411d92cd85b24)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:04.140518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:04.146675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 63/133] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/projects/865cfaca93154c65a11411d92cd85b24 => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:04.147466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:04.186003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:04.186803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.187145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.187424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.188804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:04.190264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:04.194035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 71/134] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:03 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:04.195475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:04.300942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5761b76-9ecc-4b80-9790-0d9356da0c7e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:04.317400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:04.341936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:04.342875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.343164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.343443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/c777cddf95b3464587cd81664f9af1ee/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.345735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=473c53bf5c8740e4aaabb9c9465bd165, user_id=c777cddf95b3464587cd81664f9af1ee, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:04.348181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:04.387828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 64/135] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:04 2022] PUT /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/c777cddf95b3464587cd81664f9af1ee/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:04.388414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:04.392618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32044d68-2247-4a92-98db-951b06a2d9cd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:04.432654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:04.453192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e19791-6359-4adf-af6f-39d962f69453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:04.454037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e19791-6359-4adf-af6f-39d962f69453 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.454323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e19791-6359-4adf-af6f-39d962f69453 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.454602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e19791-6359-4adf-af6f-39d962f69453 None admin] PATH_INFO: `/v3/projects/865cfaca93154c65a11411d92cd85b24/users/c777cddf95b3464587cd81664f9af1ee/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.459714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e19791-6359-4adf-af6f-39d962f69453 None admin] RBAC: Authorizing `identity:create_grant(project_id=865cfaca93154c65a11411d92cd85b24, user_id=c777cddf95b3464587cd81664f9af1ee, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:04.461209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e19791-6359-4adf-af6f-39d962f69453 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:04.490977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 72/136] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:04 2022] PUT /identity/v3/projects/865cfaca93154c65a11411d92cd85b24/users/c777cddf95b3464587cd81664f9af1ee/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:04.559304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71455f-fef0-41d8-8d5b-8fbe4ca834d4 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:04.598694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:04.599640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.599931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.600213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/c06586f3b93746d589ca131fe0aad76b/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.602429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] RBAC: Authorizing `identity:create_grant(project_id=473c53bf5c8740e4aaabb9c9465bd165, user_id=c06586f3b93746d589ca131fe0aad76b, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:04.604018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:04.605274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385502-d629-453b-bdf6-09f896e56613 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.605585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385502-d629-453b-bdf6-09f896e56613 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.605869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385502-d629-453b-bdf6-09f896e56613 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.607389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 73/137] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:04.615083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f7d1-3679-413f-99a6-f8e939c52a6a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.615405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f7d1-3679-413f-99a6-f8e939c52a6a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.615698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f7d1-3679-413f-99a6-f8e939c52a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.631776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 65/138] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:04 2022] PUT /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/c06586f3b93746d589ca131fe0aad76b/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:04.647844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6735f7d1-3679-413f-99a6-f8e939c52a6a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:04.649098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6735f7d1-3679-413f-99a6-f8e939c52a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:04.751553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 74/139] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:04.778862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:04.785635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa259b-7833-43e4-abec-0b8908860728 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.785962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa259b-7833-43e4-abec-0b8908860728 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.786196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa259b-7833-43e4-abec-0b8908860728 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.787606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 75/140] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:04.799591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1993e-6b1e-4e04-9544-50345a80c9e1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:04.799910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1993e-6b1e-4e04-9544-50345a80c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:04.800159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1993e-6b1e-4e04-9544-50345a80c9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:04.824055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:04.827230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fce1993e-6b1e-4e04-9544-50345a80c9e1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:04.828448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fce1993e-6b1e-4e04-9544-50345a80c9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:04.946005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1d21-f23c-432f-81a3-0dce1ae95665 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:04.964004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 76/141] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:04.995446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1993e-6b1e-4e04-9544-50345a80c9e1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:05.006537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:05.007288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:05.007574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:05.007853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:05.013304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:05.014783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:05.016773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 66/142] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:05.021768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:05.087168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:05.155485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1993e-6b1e-4e04-9544-50345a80c9e1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:05.218336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:05.219156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:05.219435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:05.219709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:05.225178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:05.235947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2ee32-6f07-42c7-8dbd-e36d185f1b77 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:05.241170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:05.261786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:05.262677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:05.263443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 77/143] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:05.273013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:05.273541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:05.273728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:05.273969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:05.274250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:05.275083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:05.276595 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:05.305747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 67/144] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:05 2022] POST /identity/v3/users => generated 283 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:05:05.458309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cc89e-8425-4167-be17-2cd0716fd817 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:05.475040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:05.505176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:05.505855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:05.506159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:05.506423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:05.507190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:05.507496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:05.508932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:05.520027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 78/145] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:05.525790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:05.671967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fe923-abde-4777-9716-794a3b5331ca None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:05.707237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:05.708116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:05.708397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:05.708746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] PATH_INFO: `/v3/users/691dc670ed5d4d018bd1b4fd160b8fc7` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:05.709921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] RBAC: Authorizing `identity:get_user(user_id=691dc670ed5d4d018bd1b4fd160b8fc7)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:05.711367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:05.715360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 68/146] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:05 2022] GET /identity/v3/users/691dc670ed5d4d018bd1b4fd160b8fc7 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:05.721673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:05.862507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565af5-75b4-4320-8101-a8bd116401bf None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:05.906187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efb18d-698f-40c3-89e0-c1341f56f783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:05.907066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb18d-698f-40c3-89e0-c1341f56f783 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:05.907355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb18d-698f-40c3-89e0-c1341f56f783 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:05.907637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb18d-698f-40c3-89e0-c1341f56f783 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:05.908905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efb18d-698f-40c3-89e0-c1341f56f783 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:05.910357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efb18d-698f-40c3-89e0-c1341f56f783 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:05.912066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 79/147] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:05 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:05.918677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:05.947940 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:06.066104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830e0e6-d156-4fe8-b216-ca5417d193c7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:06.107007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3184-b09e-4cca-819d-91a4deb200b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:06.107942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3184-b09e-4cca-819d-91a4deb200b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:06.108223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3184-b09e-4cca-819d-91a4deb200b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:06.108499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3184-b09e-4cca-819d-91a4deb200b0 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/691dc670ed5d4d018bd1b4fd160b8fc7/roles/f5a4836824db455d937bdc03a76f1c76` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:06.110574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3184-b09e-4cca-819d-91a4deb200b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=473c53bf5c8740e4aaabb9c9465bd165, user_id=691dc670ed5d4d018bd1b4fd160b8fc7, role_id=f5a4836824db455d937bdc03a76f1c76)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:06.112041 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3184-b09e-4cca-819d-91a4deb200b0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:06.136674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 69/148] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:05 2022] PUT /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/users/691dc670ed5d4d018bd1b4fd160b8fc7/roles/f5a4836824db455d937bdc03a76f1c76 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:06.733391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146312-2d46-4e49-bdbb-d428d62ac4e9 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:06.733922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146312-2d46-4e49-bdbb-d428d62ac4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:06.733922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146312-2d46-4e49-bdbb-d428d62ac4e9 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:06.735324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 80/149] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:06.740756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a070f-5c2b-465d-b96a-aed61c650862 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:06.741052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a070f-5c2b-465d-b96a-aed61c650862 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:06.741335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a070f-5c2b-465d-b96a-aed61c650862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:06.759241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-315a070f-5c2b-465d-b96a-aed61c650862 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:06.761437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-315a070f-5c2b-465d-b96a-aed61c650862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:06.836949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 70/150] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:06.860886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6146312-2d46-4e49-bdbb-d428d62ac4e9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:06.910246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:06.974337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6146312-2d46-4e49-bdbb-d428d62ac4e9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:07.009924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:07.011624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:07.012111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:07.012621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:07.024082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:07.033197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:07.034157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:07.035132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:07.040732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 81/151] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:07.043561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a070f-5c2b-465d-b96a-aed61c650862 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:07.058354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:07.149685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a070f-5c2b-465d-b96a-aed61c650862 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:07.185442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:07.186117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:07.186398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:07.186725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:07.187486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:07.187791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:07.189229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:07.198752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 71/152] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:07.204156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:07.311080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:07.314229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51ae0a-ecff-4cbe-8e10-fbb6e7924f3b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:07.349152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f51f2-eb34-4d29-aa31-1a6b5a4249fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:07.350117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f51f2-eb34-4d29-aa31-1a6b5a4249fb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:07.350397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f51f2-eb34-4d29-aa31-1a6b5a4249fb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:07.350674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f51f2-eb34-4d29-aa31-1a6b5a4249fb None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:07.351887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f51f2-eb34-4d29-aa31-1a6b5a4249fb None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:07.353369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f51f2-eb34-4d29-aa31-1a6b5a4249fb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:07.355042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 82/153] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:07 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:07.360486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:07.441043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:07.467326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1a940-b40f-4acc-b944-351b1e9de30a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:07.502670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:07.503334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:07.503614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:07.503888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:07.573404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:07.574802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:07.587589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4d4c-8406-424f-910b-c201c390e2db None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:07.587882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4d4c-8406-424f-910b-c201c390e2db None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:07.588145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4d4c-8406-424f-910b-c201c390e2db None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:07.589617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 83/154] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:07.594856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9743017-9ed2-484f-af7e-638132f10511 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:07.595138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9743017-9ed2-484f-af7e-638132f10511 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:07.595416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9743017-9ed2-484f-af7e-638132f10511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:07.624456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b9743017-9ed2-484f-af7e-638132f10511 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:07.625997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b9743017-9ed2-484f-af7e-638132f10511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:07.681945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:07.720662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 72/155] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:07.722117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:07.828165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae807-6ff9-4c50-811b-f6f8ec6fe047 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:07.865047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:07.865924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:07.866208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:07.866484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:07.867692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:07.869279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:07.871364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 73/156] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:07 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:07.877077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:07.902482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 84/157] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:07.938065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9743017-9ed2-484f-af7e-638132f10511 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:07.938964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:07.987540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e8b1-99c0-4c24-8a05-f9cc70774a5f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:08.048341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:08.049104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.049320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.049597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.050446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] RBAC: Adding query filter params (role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=c777cddf95b3464587cd81664f9af1ee) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:08.050757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:08.052215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:08.078701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 74/158] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:07 2022] GET /identity/v3/role_assignments?user.id=c777cddf95b3464587cd81664f9af1ee&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=cc730cc298404131a99c3547377a5e49 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:08.106563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9743017-9ed2-484f-af7e-638132f10511 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:08.122792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:08.144256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:08.145123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.145406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.145679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.157558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:08.165494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:08.166405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:08.167790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 85/159] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:08.173365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:08.294453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:08.301433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b450bf-cd06-4a50-a803-729aa69189ee None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:08.344619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:08.345232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.345515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.345794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.346563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:08.346870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:08.348290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:08.360354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 75/160] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:08.365712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:08.474726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83658517-35a1-4d33-918d-a7cd4a12a4b2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:08.511238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f84e6-87af-4d61-98ab-dd2c7b1c5fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:08.512114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f84e6-87af-4d61-98ab-dd2c7b1c5fab None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.512398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f84e6-87af-4d61-98ab-dd2c7b1c5fab None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.512758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f84e6-87af-4d61-98ab-dd2c7b1c5fab None admin] PATH_INFO: `/v3/users/c06586f3b93746d589ca131fe0aad76b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.514073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f84e6-87af-4d61-98ab-dd2c7b1c5fab None admin] RBAC: Authorizing `identity:get_user(user_id=c06586f3b93746d589ca131fe0aad76b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:08.515555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f84e6-87af-4d61-98ab-dd2c7b1c5fab None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:08.517571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 86/161] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:08 2022] GET /identity/v3/users/c06586f3b93746d589ca131fe0aad76b => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:08.523042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:08.537587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e413-4d94-4d84-b392-48de33b54952 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.537877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e413-4d94-4d84-b392-48de33b54952 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.538205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e413-4d94-4d84-b392-48de33b54952 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.539647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 87/162] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:08.553962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec29b1-f232-4f7b-a3a8-2fbe01dbcb3f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.554254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec29b1-f232-4f7b-a3a8-2fbe01dbcb3f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.554520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec29b1-f232-4f7b-a3a8-2fbe01dbcb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.571930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ceec29b1-f232-4f7b-a3a8-2fbe01dbcb3f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:08.574215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ceec29b1-f232-4f7b-a3a8-2fbe01dbcb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:08.597121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:08.646237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 88/163] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:08.647995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c636c-a639-4662-b09b-9732da641c40 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.648481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c636c-a639-4662-b09b-9732da641c40 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.649041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c636c-a639-4662-b09b-9732da641c40 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.650511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 89/164] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:08.657874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ab1e4-29ad-4940-a9d0-e0e9d0145c4e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.658905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ab1e4-29ad-4940-a9d0-e0e9d0145c4e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.659387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ab1e4-29ad-4940-a9d0-e0e9d0145c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.678529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7b6ab1e4-29ad-4940-a9d0-e0e9d0145c4e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:08.680070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7b6ab1e4-29ad-4940-a9d0-e0e9d0145c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:08.708963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d753d80-fab2-4192-b2b9-27c28a5c8e57 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:08.758551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 90/165] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:08.759766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ab1e4-29ad-4940-a9d0-e0e9d0145c4e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:08.790940 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:08.792044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:08.792532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:08.793035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:08.858818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:08.903421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:08.935697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ab1e4-29ad-4940-a9d0-e0e9d0145c4e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:08.942730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:08.944465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:09.037822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:09.038843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.039270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.039668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.077444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:09.089494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:09.090400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:09.091392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:09.093944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 91/166] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:09.094922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:09.152666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 76/167] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:09.154313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:09.258541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:09.273510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe17525-7186-472a-916b-29c6fcb89b51 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:09.310136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:09.311246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.312075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.312539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.325285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:09.334196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:09.335077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:09.335850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:09.339046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 92/168] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:09.339291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f7d2f-0c76-4c1a-90f6-b9eaccb49af8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:09.340228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:09.359244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:09.430769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:09.431376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.431681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.431939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.433047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:09.433356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:09.434851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:09.451800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 77/169] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:09.452695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:09.608244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad715c-94e5-431a-9c89-c1c8ef658b62 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:09.645483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06f89-ff31-44c3-870b-a3c1597f913a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:09.660257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:09.661147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.661429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.661702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.662919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:09.664370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:09.666200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 93/170] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:09.667703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:09.699887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:09.721606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:09.722508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.724884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.725312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.726210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:09.727365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:09.728916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:09.745497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 78/171] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:09.747935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:09.808932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:09.851282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8b28-82fe-4bd4-857e-916d15504cd0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:09.905983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbb11a-5452-4be2-811c-cb18d4c6dbae None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:09.939468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46266938-8ea5-4c6e-b4c5-cb0a43e9b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:09.941494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46266938-8ea5-4c6e-b4c5-cb0a43e9b19b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.942041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46266938-8ea5-4c6e-b4c5-cb0a43e9b19b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.942612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46266938-8ea5-4c6e-b4c5-cb0a43e9b19b None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.945168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46266938-8ea5-4c6e-b4c5-cb0a43e9b19b None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:09.947678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46266938-8ea5-4c6e-b4c5-cb0a43e9b19b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:09.952275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 94/172] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:09.954411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66030f4-8945-4d9f-ac54-31af54999feb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.955373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66030f4-8945-4d9f-ac54-31af54999feb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.955939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66030f4-8945-4d9f-ac54-31af54999feb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.957854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 95/173] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:09.959057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66030f4-8945-4d9f-ac54-31af54999feb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:09.964996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:09.965856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:09.966275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:09.966709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:09.967683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=c06586f3b93746d589ca131fe0aad76b) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:09.968133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:09.969900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:09.984283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 79/174] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/role_assignments?user.id=c06586f3b93746d589ca131fe0aad76b&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:09.986838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:10.110292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:10.206531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66030f4-8945-4d9f-ac54-31af54999feb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:10.227910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62ef73-5c10-44a7-882d-03d204d92c44 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:10.259526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d7100-20f5-41db-9c37-4ee027ad1081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:10.261599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d7100-20f5-41db-9c37-4ee027ad1081 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.261892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d7100-20f5-41db-9c37-4ee027ad1081 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.262334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d7100-20f5-41db-9c37-4ee027ad1081 None admin] PATH_INFO: `/v3/users/c777cddf95b3464587cd81664f9af1ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.263690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d7100-20f5-41db-9c37-4ee027ad1081 None admin] RBAC: Authorizing `identity:get_user(user_id=c777cddf95b3464587cd81664f9af1ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:10.265393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d7100-20f5-41db-9c37-4ee027ad1081 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:10.267237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 96/175] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/users/c777cddf95b3464587cd81664f9af1ee => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:10.268678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb301a6-d373-4025-a644-8fea30aee4f0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.269176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb301a6-d373-4025-a644-8fea30aee4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.269652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb301a6-d373-4025-a644-8fea30aee4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.273725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dc6f7-7c00-4d09-8553-3bb44e1fe8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:10.274383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc6f7-7c00-4d09-8553-3bb44e1fe8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.274666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc6f7-7c00-4d09-8553-3bb44e1fe8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.274939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc6f7-7c00-4d09-8553-3bb44e1fe8c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.289051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4bb301a6-d373-4025-a644-8fea30aee4f0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:10.290605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4bb301a6-d373-4025-a644-8fea30aee4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:10.295957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:10.372018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2dc6f7-7c00-4d09-8553-3bb44e1fe8c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:10.372423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 97/176] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:10.373703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2dc6f7-7c00-4d09-8553-3bb44e1fe8c3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:10.374188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e1acb-d704-41c2-9726-24ffebabb1a5 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.374701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e1acb-d704-41c2-9726-24ffebabb1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.375374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e1acb-d704-41c2-9726-24ffebabb1a5 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.377032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 98/177] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:10.377958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e1acb-d704-41c2-9726-24ffebabb1a5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:10.440306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:10.498540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e1acb-d704-41c2-9726-24ffebabb1a5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:10.505115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 80/178] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:10.506662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0b035-65f1-4f7b-a353-a51c88fbfc32 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.506953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0b035-65f1-4f7b-a353-a51c88fbfc32 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.507229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0b035-65f1-4f7b-a353-a51c88fbfc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.523300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-94a0b035-65f1-4f7b-a353-a51c88fbfc32 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:10.524540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-94a0b035-65f1-4f7b-a353-a51c88fbfc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:10.534454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c7675-7515-4d4f-804d-5c1efbfe1e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:10.535107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7675-7515-4d4f-804d-5c1efbfe1e3f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.535388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7675-7515-4d4f-804d-5c1efbfe1e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.535664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7675-7515-4d4f-804d-5c1efbfe1e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.546731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:10.595703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 81/179] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:10.596782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0b035-65f1-4f7b-a353-a51c88fbfc32 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:10.605344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c7675-7515-4d4f-804d-5c1efbfe1e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:10.606798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c7675-7515-4d4f-804d-5c1efbfe1e3f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:10.686104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:10.705828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0b035-65f1-4f7b-a353-a51c88fbfc32 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:10.706699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 99/180] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:10.708000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e9b46-1330-49a6-ba7a-fc56ab416dd0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.708290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e9b46-1330-49a6-ba7a-fc56ab416dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.708586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e9b46-1330-49a6-ba7a-fc56ab416dd0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.709853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 100/181] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:10.710626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e9b46-1330-49a6-ba7a-fc56ab416dd0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:10.746835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72104836-303d-4933-9389-ede53540847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:10.747630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72104836-303d-4933-9389-ede53540847a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.747906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72104836-303d-4933-9389-ede53540847a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.748178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72104836-303d-4933-9389-ede53540847a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.755272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:10.763207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72104836-303d-4933-9389-ede53540847a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-72104836-303d-4933-9389-ede53540847a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:10.771722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:10.772854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:10.773808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 82/182] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:10.774455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f02039-7cdb-430d-9d4c-cb54acff21e8 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.774645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f02039-7cdb-430d-9d4c-cb54acff21e8 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.774894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f02039-7cdb-430d-9d4c-cb54acff21e8 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.776336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 83/183] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:10.780742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f02039-7cdb-430d-9d4c-cb54acff21e8 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:10.822020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e9b46-1330-49a6-ba7a-fc56ab416dd0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:10.855529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c59f99-3681-49a3-9a12-df61627c5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:10.856380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c59f99-3681-49a3-9a12-df61627c5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.856695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c59f99-3681-49a3-9a12-df61627c5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.856977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c59f99-3681-49a3-9a12-df61627c5ef8 None admin] PATH_INFO: `/v3/projects/865cfaca93154c65a11411d92cd85b24` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.858159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c59f99-3681-49a3-9a12-df61627c5ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=865cfaca93154c65a11411d92cd85b24)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:10.859608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c59f99-3681-49a3-9a12-df61627c5ef8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:10.861324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 101/184] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:10 2022] GET /identity/v3/projects/865cfaca93154c65a11411d92cd85b24 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:10.865827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374d2c3-fa7d-4c12-852c-117351baddde None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.866114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374d2c3-fa7d-4c12-852c-117351baddde None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.866394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374d2c3-fa7d-4c12-852c-117351baddde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.882096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c374d2c3-fa7d-4c12-852c-117351baddde None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:10.883269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c374d2c3-fa7d-4c12-852c-117351baddde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:10.885149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f02039-7cdb-430d-9d4c-cb54acff21e8 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:10.918679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:10.919469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:10.919759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:10.920040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:10.927515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:10.931289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:10.939601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:10.940489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:10.941295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 84/185] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:10.942436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:10.953613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 102/186] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:10.954299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374d2c3-fa7d-4c12-852c-117351baddde None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:10.996191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.054109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cdd48-f84c-4b11-bb2e-0b119859c5b3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.061174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374d2c3-fa7d-4c12-852c-117351baddde None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.087063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d9f1-7b48-4314-bd8c-8b6cf5cfe28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.087924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d9f1-7b48-4314-bd8c-8b6cf5cfe28d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.088210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d9f1-7b48-4314-bd8c-8b6cf5cfe28d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.089030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d9f1-7b48-4314-bd8c-8b6cf5cfe28d None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.090217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d9f1-7b48-4314-bd8c-8b6cf5cfe28d None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.091692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d9f1-7b48-4314-bd8c-8b6cf5cfe28d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.093408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 85/187] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:10 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.094533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15adfdb-1810-4e77-9977-dc2e39729302 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.094855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.095126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15adfdb-1810-4e77-9977-dc2e39729302 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.095126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15adfdb-1810-4e77-9977-dc2e39729302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.095518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.095799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.096073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.096853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:11.097156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.098506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.107511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 103/188] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.108626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.111834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c15adfdb-1810-4e77-9977-dc2e39729302 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:11.113050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c15adfdb-1810-4e77-9977-dc2e39729302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:11.182560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.188728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 86/189] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:11.189556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15adfdb-1810-4e77-9977-dc2e39729302 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.211545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.215069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfca1c-3c3e-483a-895f-17d83c7a574f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.248299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.248999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.249280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.249555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.250377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=865cfaca93154c65a11411d92cd85b24, user.id=c777cddf95b3464587cd81664f9af1ee) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:11.250682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.252104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.257815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 104/190] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/role_assignments?user.id=c777cddf95b3464587cd81664f9af1ee&scope.project.id=865cfaca93154c65a11411d92cd85b24&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.258576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.299616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15adfdb-1810-4e77-9977-dc2e39729302 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.337133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.337822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.338109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.338425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.339713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:11.340024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.341554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.352314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 87/191] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.353216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.369146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078b330-c7e6-470b-bbc8-62b2f55955aa None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.407517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.408316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.408681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.408976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.410199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.411507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.413449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 105/192] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.414219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.420875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.454511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.478550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c20d9-19c8-4fb6-93c9-093e06de48e4 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.513868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.514530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.514811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.515091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.515921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=c777cddf95b3464587cd81664f9af1ee) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:11.516232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.517786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.523517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 88/193] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/role_assignments?user.id=c777cddf95b3464587cd81664f9af1ee&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.525401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.530376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4935e47-0406-4b16-99bf-59b987a847d0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.572300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.573701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.574268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.574268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] PATH_INFO: `/v3/users/9322aac686b74502b989a426d4dd86c4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.591091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] RBAC: Authorizing `identity:get_user(user_id=9322aac686b74502b989a426d4dd86c4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.592781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.599530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 106/194] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/users/9322aac686b74502b989a426d4dd86c4 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.599530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.646677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba39eb-d6fc-4888-a615-5e954eecbca3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.671730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.688661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.689463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.689748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.690027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.701340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.702217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:11.712153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:11.713114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:11.713926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 89/195] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:11.714566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.716841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d187735-f82d-40ff-ba2d-024f988e5bee None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.756823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.757720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.758068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.758263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] PATH_INFO: `/v3/users/691dc670ed5d4d018bd1b4fd160b8fc7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.759469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] RBAC: Authorizing `identity:get_user(user_id=691dc670ed5d4d018bd1b4fd160b8fc7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.761532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.763176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 107/196] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/users/691dc670ed5d4d018bd1b4fd160b8fc7 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:11.763928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.825668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68188-2ac4-4c59-83f3-8724339955c5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.858172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.858842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.859124 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.859400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.860107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:11.861568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:11.866741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467925e-dfef-487a-8abc-7bfb06ddbf8e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:11.882489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 90/197] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:11 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:11.883382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:11.914665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:11.915603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:11.915896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:11.916180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:11.933196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.947411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:11.998707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.000208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.008250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50ccdb-9cef-4d87-973f-09e1333d8589 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.051579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.052261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.052544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.052911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.053630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:12.054360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.055834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.065369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 91/198] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:12.070597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.130449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 108/199] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:12.131225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.204735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d94b1-25f1-48ad-bcd0-f1eb241e696f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.206521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:12.208688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:12.247621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.248279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.248613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.248885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.256546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe037-256c-4471-b496-d5c42d20c4cd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.295839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.297114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.297682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.298160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] PATH_INFO: `/v3/users/9322aac686b74502b989a426d4dd86c4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.299704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] RBAC: Authorizing `identity:get_user(user_id=9322aac686b74502b989a426d4dd86c4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.301491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.303456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 109/200] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/users/9322aac686b74502b989a426d4dd86c4 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:12.304666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.332889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.334821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.422776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0721910-a6a4-42c7-b5ba-af1e760278a1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.456975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:12.464196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 92/201] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:12.464196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.466956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:12.475553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.477145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.478059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.478909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.481079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.482542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.489260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 110/202] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:12.489950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.584139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391a83-2e8c-48e3-920c-af1ee82cccd2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.606401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35524a6b-2554-4546-bee1-e16e5473a1c7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.620334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.621017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.621300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.621629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.643901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.645126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.645614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.646092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.647519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.649264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.651775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 111/203] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:12.652852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.693141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.694596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.696445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:12.718880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:12.761019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96a860-6f03-4052-853f-ea7a2cb9e1d0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.798439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.799473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.799993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.800471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.801528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=807a635f1d50461091b40698766b4912, user.id=9322aac686b74502b989a426d4dd86c4) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:12.802056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.803699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.804181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 93/204] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:12.805042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.811256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 112/205] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/role_assignments?user.id=9322aac686b74502b989a426d4dd86c4&scope.project.id=807a635f1d50461091b40698766b4912&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:12.812317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.923741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34138982-8f81-4ac6-b987-d7611806c294 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.953497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:12.962658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:12.964503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:12.965077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:12.965559 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:12.967075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:12.969470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:12.971847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 113/206] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:12.981875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:12.986591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2841f1-d4bf-4578-bc17-cafc769f1d23 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:12.989748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:13.057705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9713-4b8e-45f6-9df4-6972784afbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:13.058392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9713-4b8e-45f6-9df4-6972784afbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:13.058675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9713-4b8e-45f6-9df4-6972784afbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:13.058952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9713-4b8e-45f6-9df4-6972784afbb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:13.059804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b9713-4b8e-45f6-9df4-6972784afbb3 None admin] RBAC: Adding query filter params (role.id=f5a4836824db455d937bdc03a76f1c76, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165, user.id=691dc670ed5d4d018bd1b4fd160b8fc7) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:13.060119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b9713-4b8e-45f6-9df4-6972784afbb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:13.061672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b9713-4b8e-45f6-9df4-6972784afbb3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:13.071389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 94/207] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/role_assignments?user.id=691dc670ed5d4d018bd1b4fd160b8fc7&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=f5a4836824db455d937bdc03a76f1c76 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:13.151649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cad23-cd57-4353-83dd-8d823d87d2ee None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:13.208117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2258aa-188a-4eac-bc0d-d8341cbf36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:13.208752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2258aa-188a-4eac-bc0d-d8341cbf36ec None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:13.209041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2258aa-188a-4eac-bc0d-d8341cbf36ec None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:13.209320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2258aa-188a-4eac-bc0d-d8341cbf36ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:13.210188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2258aa-188a-4eac-bc0d-d8341cbf36ec None admin] RBAC: Adding query filter params (role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=807a635f1d50461091b40698766b4912, user.id=9322aac686b74502b989a426d4dd86c4) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:13.210498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2258aa-188a-4eac-bc0d-d8341cbf36ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:13.211976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2258aa-188a-4eac-bc0d-d8341cbf36ec None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:13.221228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 114/208] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:12 2022] GET /identity/v3/role_assignments?user.id=9322aac686b74502b989a426d4dd86c4&scope.project.id=807a635f1d50461091b40698766b4912&role.id=cc730cc298404131a99c3547377a5e49 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:14.167606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0325-7d9a-4c93-acc9-05cf59dca139 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.168302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0325-7d9a-4c93-acc9-05cf59dca139 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.168302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0325-7d9a-4c93-acc9-05cf59dca139 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.169574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 95/209] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:14.175021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214977be-e58a-4c80-8697-9f95ee533f33 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.175777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214977be-e58a-4c80-8697-9f95ee533f33 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.176201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214977be-e58a-4c80-8697-9f95ee533f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.193624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-214977be-e58a-4c80-8697-9f95ee533f33 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:14.195011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-214977be-e58a-4c80-8697-9f95ee533f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:14.250094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:14.272658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 115/210] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:14.279272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbd012-6661-4da3-84f7-8a6fd70b4b58 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.279571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbd012-6661-4da3-84f7-8a6fd70b4b58 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.279851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbd012-6661-4da3-84f7-8a6fd70b4b58 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.281282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 96/211] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:14.286659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a34115-bc46-4a95-9fd1-49a191031014 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.287114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a34115-bc46-4a95-9fd1-49a191031014 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.287536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a34115-bc46-4a95-9fd1-49a191031014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.291857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbd012-6661-4da3-84f7-8a6fd70b4b58 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:14.305118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-24a34115-bc46-4a95-9fd1-49a191031014 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:14.306483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-24a34115-bc46-4a95-9fd1-49a191031014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:14.377945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 116/212] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:14.399338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbd012-6661-4da3-84f7-8a6fd70b4b58 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:14.403868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a34115-bc46-4a95-9fd1-49a191031014 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:14.446113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:14.447289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.448677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.449153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.462498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:14.473360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:14.474249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:14.475027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:14.477811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 97/213] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:14.485611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:14.517121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a34115-bc46-4a95-9fd1-49a191031014 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:14.525252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:14.547869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:14.570708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:14.573205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.576891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.577386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.579380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:14.581744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:14.584055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 117/214] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:14.589974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:14.609437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339931a3-0113-4468-8ee5-45d363ded5d0 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:14.645296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:14.646024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.646608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.646717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.647465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:14.647816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:14.649214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:14.658022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 98/215] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:14.663797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:14.717263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1d9c9-3228-43d4-a06e-3c0a23a496f3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:14.762969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:14.768826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:14.769889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.770263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4010a3c-1738-4e00-99f3-cf1b4a65faac None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:14.770778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.771202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.772074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:14.775677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:14.808665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 118/216] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:14 2022] POST /identity/v3/users => generated 297 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:14.819715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ebb03-796c-4a9f-89ad-db691c4005c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:14.820757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ebb03-796c-4a9f-89ad-db691c4005c0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:14.821098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ebb03-796c-4a9f-89ad-db691c4005c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:14.821387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ebb03-796c-4a9f-89ad-db691c4005c0 None admin] PATH_INFO: `/v3/users/09dc8be2f93b42bbbddb6b46e1c23a12` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:14.841240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ebb03-796c-4a9f-89ad-db691c4005c0 None admin] RBAC: Authorizing `identity:get_user(user_id=09dc8be2f93b42bbbddb6b46e1c23a12)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:14.842693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ebb03-796c-4a9f-89ad-db691c4005c0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:14.849512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 99/217] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:14 2022] GET /identity/v3/users/09dc8be2f93b42bbbddb6b46e1c23a12 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:14.850966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:14.912970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:14.980002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224668-a6f7-4a6c-8b49-b27f0a22567f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:15.017587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:15.018251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.018536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.018813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.098136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8335c32-81d9-4f86-b59e-7dc7f633d910 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.098968 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8335c32-81d9-4f86-b59e-7dc7f633d910 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.099826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8335c32-81d9-4f86-b59e-7dc7f633d910 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.101780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 100/218] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:15.108102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465889b-6d04-4bf8-8337-afb52943d4ab None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.108378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:15.108486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465889b-6d04-4bf8-8337-afb52943d4ab None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.108691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465889b-6d04-4bf8-8337-afb52943d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.109905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:15.134511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1465889b-6d04-4bf8-8337-afb52943d4ab None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:15.136352 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1465889b-6d04-4bf8-8337-afb52943d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:15.162651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:15.234601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 101/219] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:15.242375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 119/220] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:15.248195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465889b-6d04-4bf8-8337-afb52943d4ab None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:15.262069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:15.344020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:15.370464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465889b-6d04-4bf8-8337-afb52943d4ab None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:15.410306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d153d06-776c-4fe3-b019-e03fa3d98a98 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:15.423203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c0115-d552-429b-acdf-01cba53d16eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:15.424422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c0115-d552-429b-acdf-01cba53d16eb None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.425011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c0115-d552-429b-acdf-01cba53d16eb None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.425513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c0115-d552-429b-acdf-01cba53d16eb None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.426970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c0115-d552-429b-acdf-01cba53d16eb None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:15.428758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c0115-d552-429b-acdf-01cba53d16eb None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:15.436132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 102/221] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:15.437771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145e58-4d8d-404b-90af-e489ecbc3a9c None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.438366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145e58-4d8d-404b-90af-e489ecbc3a9c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.438839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145e58-4d8d-404b-90af-e489ecbc3a9c None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.440393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 103/222] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:15.441983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13145e58-4d8d-404b-90af-e489ecbc3a9c None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:15.448340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42702f9b-0649-4304-82f6-3658ef944896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:15.449176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702f9b-0649-4304-82f6-3658ef944896 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.449870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702f9b-0649-4304-82f6-3658ef944896 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.450143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702f9b-0649-4304-82f6-3658ef944896 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.463340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42702f9b-0649-4304-82f6-3658ef944896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-42702f9b-0649-4304-82f6-3658ef944896 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:15.473097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:15.473975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 120/223] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:15.478683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560520-3a42-4872-9a6c-365a8a8ca20c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.479446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560520-3a42-4872-9a6c-365a8a8ca20c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.479446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560520-3a42-4872-9a6c-365a8a8ca20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.483349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:15.500342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4e560520-3a42-4872-9a6c-365a8a8ca20c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:15.502065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4e560520-3a42-4872-9a6c-365a8a8ca20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:15.570196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13145e58-4d8d-404b-90af-e489ecbc3a9c None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:15.593428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 121/224] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:15.594586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e560520-3a42-4872-9a6c-365a8a8ca20c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:15.629177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:15.630161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.630511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.630860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.632067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=807a635f1d50461091b40698766b4912, user.id=09dc8be2f93b42bbbddb6b46e1c23a12) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:15.632383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:15.634036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:15.642350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 104/225] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/role_assignments?user.id=09dc8be2f93b42bbbddb6b46e1c23a12&scope.project.id=807a635f1d50461091b40698766b4912&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:15.643011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:15.672883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:15.713176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e560520-3a42-4872-9a6c-365a8a8ca20c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:15.749010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:15.754011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:15.754797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.755159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.755508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.756486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:15.757011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:15.758510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:15.768656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 122/226] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:15.776661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:15.864513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7339a-1856-4383-bfa4-1855c74b00ad None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:15.912372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:15.913513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.913796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.914070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.928215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:15.928635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee57bc-15c5-4159-9685-d70e8a55edbb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:15.939464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:15.940361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:15.941205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:15.942480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 105/227] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:15.947514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:15.968411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:15.969778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:15.970290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:15.970799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] PATH_INFO: `/v3/users/9322aac686b74502b989a426d4dd86c4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:15.973108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=9322aac686b74502b989a426d4dd86c4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:15.975544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:15.977581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 123/228] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/users/9322aac686b74502b989a426d4dd86c4 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:15.983063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:16.020824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:16.062841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1d060-a9f2-4b58-8ee4-7758ae199baa None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:16.078859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:16.101019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:16.101704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:16.101986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:16.102258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:16.103006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:16.103406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:16.105196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:16.114191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 106/229] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:16.120002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:16.158814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fe7d5-0e9d-401d-bbbe-ebb361ab3ae6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:16.219489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:16.220474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:16.220848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:16.221192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:16.224229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:16.225748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:16.227460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 124/230] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:15 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:16.233114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:16.272415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111b4d5-4ec0-4806-bfc6-59c9c7696ec6 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:16.305743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:16.317788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:16.320215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:16.320494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:16.320794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] PATH_INFO: `/v3/users/9322aac686b74502b989a426d4dd86c4` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:16.322163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] RBAC: Authorizing `identity:get_user(user_id=9322aac686b74502b989a426d4dd86c4)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:16.328795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:16.330555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 107/231] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:16 2022] GET /identity/v3/users/9322aac686b74502b989a426d4dd86c4 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:16.337853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:16.392730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0adc30-d834-45a7-9650-6c17f47858ed None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:16.421931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:16.435263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:16.436143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:16.436426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:16.436745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912/users/9322aac686b74502b989a426d4dd86c4/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:16.440167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] RBAC: Authorizing `identity:create_grant(project_id=807a635f1d50461091b40698766b4912, user_id=9322aac686b74502b989a426d4dd86c4, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:16.441641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:16.462285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 125/232] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:16 2022] PUT /identity/v3/projects/807a635f1d50461091b40698766b4912/users/9322aac686b74502b989a426d4dd86c4/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:05:16.536211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0877cc4-bc5d-42e2-9cf4-eff4f28b0db1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:16.593330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c950bf9-a5ed-433a-8ed8-b6f78f4f1845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:16.594218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c950bf9-a5ed-433a-8ed8-b6f78f4f1845 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:16.594500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c950bf9-a5ed-433a-8ed8-b6f78f4f1845 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:16.594776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c950bf9-a5ed-433a-8ed8-b6f78f4f1845 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:16.596107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c950bf9-a5ed-433a-8ed8-b6f78f4f1845 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:16.597620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c950bf9-a5ed-433a-8ed8-b6f78f4f1845 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:16.599543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 108/233] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:16 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:16.605451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:16.718281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d9908-695b-48ec-a785-756d1fa9fb4e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:16.754488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5942a0-0ae8-4a5f-9d84-edaf9512ff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:16.755391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5942a0-0ae8-4a5f-9d84-edaf9512ff25 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:16.755675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5942a0-0ae8-4a5f-9d84-edaf9512ff25 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:16.755951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5942a0-0ae8-4a5f-9d84-edaf9512ff25 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912/users/9322aac686b74502b989a426d4dd86c4/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:16.758364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5942a0-0ae8-4a5f-9d84-edaf9512ff25 None admin] RBAC: Authorizing `identity:create_grant(project_id=807a635f1d50461091b40698766b4912, user_id=9322aac686b74502b989a426d4dd86c4, role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:16.759816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5942a0-0ae8-4a5f-9d84-edaf9512ff25 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:16.776587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 126/234] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:16 2022] PUT /identity/v3/projects/807a635f1d50461091b40698766b4912/users/9322aac686b74502b989a426d4dd86c4/roles/cc730cc298404131a99c3547377a5e49 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:05:17.412632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904bd76-0571-4fbb-a1d2-3400c2102280 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:17.413122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904bd76-0571-4fbb-a1d2-3400c2102280 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:17.413122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904bd76-0571-4fbb-a1d2-3400c2102280 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:17.414476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 109/235] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:17.419805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74cf2a-cf42-4771-a5b9-ce405299f6c1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:17.420263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74cf2a-cf42-4771-a5b9-ce405299f6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:17.420440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74cf2a-cf42-4771-a5b9-ce405299f6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:17.445646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4b74cf2a-cf42-4771-a5b9-ce405299f6c1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:17.446889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4b74cf2a-cf42-4771-a5b9-ce405299f6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:17.506773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:17.539792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 127/236] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:17.574694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904bd76-0571-4fbb-a1d2-3400c2102280 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:17.636358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ec079-8751-4239-9c29-90123eac08d0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:17.636713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ec079-8751-4239-9c29-90123eac08d0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:17.636960 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ec079-8751-4239-9c29-90123eac08d0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:17.638358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 128/237] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:17.647438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1142bc-8380-48cf-acba-b3be7250b225 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:17.647730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1142bc-8380-48cf-acba-b3be7250b225 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:17.648010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1142bc-8380-48cf-acba-b3be7250b225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:17.675577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2f1142bc-8380-48cf-acba-b3be7250b225 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:17.676714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2f1142bc-8380-48cf-acba-b3be7250b225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:17.680889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:17.715792 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904bd76-0571-4fbb-a1d2-3400c2102280 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:17.781435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:17.782235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:17.782512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:17.782781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:17.800716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:17.810726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 129/238] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:17.810867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:17.812006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:17.813009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 110/239] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:17.826803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1142bc-8380-48cf-acba-b3be7250b225 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:17.842373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:18.004193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1142bc-8380-48cf-acba-b3be7250b225 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.016894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:18.030156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668117d-c762-427b-9c05-f3148b9dd2d0 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.084948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:18.086016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.086834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.087120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.087745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.088649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.088933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.089209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.089555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.090009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:18.090353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:18.092044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:18.092959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:18.095001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:18.097228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 111/240] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:18.103094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:18.115129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 130/241] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:18.121010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:18.264916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31029c-cd6d-4dc1-9b18-c88f81b5b9df None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.301367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0067d0-c11d-40d1-8062-ca8cc4b85b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.302058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0067d0-c11d-40d1-8062-ca8cc4b85b3f None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.302346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0067d0-c11d-40d1-8062-ca8cc4b85b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.302628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0067d0-c11d-40d1-8062-ca8cc4b85b3f None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.303350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0067d0-c11d-40d1-8062-ca8cc4b85b3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:18.303832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef6a4b-dd25-45cb-9206-f88a9eae7a34 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.304906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0067d0-c11d-40d1-8062-ca8cc4b85b3f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:18.330797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 112/242] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:18 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:05:18.336427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10273523-71d2-49f4-a5cf-aa58c91e5341 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.336427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10273523-71d2-49f4-a5cf-aa58c91e5341 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.336427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10273523-71d2-49f4-a5cf-aa58c91e5341 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.336427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 113/243] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:18.346199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf9e83-dba2-4cdf-8b74-e0586b1ea9df None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.346505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf9e83-dba2-4cdf-8b74-e0586b1ea9df None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.346766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf9e83-dba2-4cdf-8b74-e0586b1ea9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.367173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1acf9e83-dba2-4cdf-8b74-e0586b1ea9df None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:18.368400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1acf9e83-dba2-4cdf-8b74-e0586b1ea9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:18.369282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.370232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.370516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.370793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] PATH_INFO: `/v3/users/fb54706bdfaf49e8b3e13fbe7177b602` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.401598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] RBAC: Authorizing `identity:get_user(user_id=fb54706bdfaf49e8b3e13fbe7177b602)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:18.403105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:18.407965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 131/244] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:18 2022] GET /identity/v3/users/fb54706bdfaf49e8b3e13fbe7177b602 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:18.413991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:18.460442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:18.480582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:18.514744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 114/245] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:18.546764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27462a3-4c29-4284-a256-d5ad34a12666 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.552259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf9e83-dba2-4cdf-8b74-e0586b1ea9df None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:18.589917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.590580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.590864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.591141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.675254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:18.677253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:18.694642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf9e83-dba2-4cdf-8b74-e0586b1ea9df None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.712707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:18.742872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.743795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.744169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.744647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.755532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:18.769466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:18.770411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:18.771465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 115/246] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:18.777709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:18.813312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 132/247] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:18.819772 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:18.841834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:18.914164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddad8d-a4a4-4b27-a13d-152185cff5ea None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.943536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1077ba-131e-42f7-8e0a-b16cac442c79 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:18.958704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cf7e1-09b9-4c45-8460-37988058aa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.959400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf7e1-09b9-4c45-8460-37988058aa86 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.959689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf7e1-09b9-4c45-8460-37988058aa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.959972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf7e1-09b9-4c45-8460-37988058aa86 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.960874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cf7e1-09b9-4c45-8460-37988058aa86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:18.961188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cf7e1-09b9-4c45-8460-37988058aa86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:18.962575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cf7e1-09b9-4c45-8460-37988058aa86 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:18.976392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 116/248] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:18.977652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfdd9e-248b-45c2-8d26-9dad7936d647 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.977942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfdd9e-248b-45c2-8d26-9dad7936d647 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.978218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfdd9e-248b-45c2-8d26-9dad7936d647 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.979996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 117/249] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:18.984615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:18.986362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae85f74-f5bc-4a6a-b406-9dae71b743fe None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.986480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae85f74-f5bc-4a6a-b406-9dae71b743fe None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.986716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae85f74-f5bc-4a6a-b406-9dae71b743fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.995460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90570155-5599-4646-962b-a52007c41be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:18.996596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90570155-5599-4646-962b-a52007c41be7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:18.996724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90570155-5599-4646-962b-a52007c41be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:18.996990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90570155-5599-4646-962b-a52007c41be7 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:18.998846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90570155-5599-4646-962b-a52007c41be7 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:19.000339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90570155-5599-4646-962b-a52007c41be7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:19.002509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 133/250] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:18 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:19.003290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90570155-5599-4646-962b-a52007c41be7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:19.009644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-cae85f74-f5bc-4a6a-b406-9dae71b743fe None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:19.011661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cae85f74-f5bc-4a6a-b406-9dae71b743fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:19.097507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 118/251] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:19.099086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae85f74-f5bc-4a6a-b406-9dae71b743fe None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:19.141212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:19.162588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90570155-5599-4646-962b-a52007c41be7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:19.226282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae85f74-f5bc-4a6a-b406-9dae71b743fe None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:19.250219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:19.251117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.251405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.251683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] PATH_INFO: `/v3/users/09dc8be2f93b42bbbddb6b46e1c23a12` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.257304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] RBAC: Authorizing `identity:get_user(user_id=09dc8be2f93b42bbbddb6b46e1c23a12)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:19.258785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:19.261092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 134/252] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:19 2022] GET /identity/v3/users/09dc8be2f93b42bbbddb6b46e1c23a12 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:19.261938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:19.271739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:19.272800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.273285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.273759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.274798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] RBAC: Adding query filter params (role.id=f5a4836824db455d937bdc03a76f1c76, scope.project.id=807a635f1d50461091b40698766b4912, user.id=fb54706bdfaf49e8b3e13fbe7177b602) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:19.275311 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:19.277071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:19.283841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 119/253] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:19 2022] GET /identity/v3/role_assignments?user.id=fb54706bdfaf49e8b3e13fbe7177b602&scope.project.id=807a635f1d50461091b40698766b4912&role.id=f5a4836824db455d937bdc03a76f1c76 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:19.285075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:19.391577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:19.398480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdaac-6121-4d94-826a-112784d77604 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:19.436243 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:19.437524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.438009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.438518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.439909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:19.441589 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:19.443504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 120/254] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:19 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:19.449235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:19.450415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:19.528442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c434846-d2da-4dce-a0f3-1d0fe2c7da86 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:19.581469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5b0cd-fb48-49ac-a5c3-b6817cbed165 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:19.585261 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4cc9b-91bc-4b39-8e3b-171d1424c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:19.586309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4cc9b-91bc-4b39-8e3b-171d1424c009 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.586789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4cc9b-91bc-4b39-8e3b-171d1424c009 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.587264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4cc9b-91bc-4b39-8e3b-171d1424c009 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.600748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4cc9b-91bc-4b39-8e3b-171d1424c009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7ec4cc9b-91bc-4b39-8e3b-171d1424c009 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:19.610517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:19.611403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:19.612200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:19.615180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 135/255] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:19.616821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a53e2-cead-4c79-a005-e2491359dc08 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.617289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a53e2-cead-4c79-a005-e2491359dc08 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.617763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a53e2-cead-4c79-a005-e2491359dc08 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.619484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bf394-0d4d-45e0-ad1d-ebfaf75c70fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:19.619696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 136/256] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:19.620855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf394-0d4d-45e0-ad1d-ebfaf75c70fd None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.621220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf394-0d4d-45e0-ad1d-ebfaf75c70fd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.621412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a53e2-cead-4c79-a005-e2491359dc08 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:19.621623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf394-0d4d-45e0-ad1d-ebfaf75c70fd None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912/users/09dc8be2f93b42bbbddb6b46e1c23a12/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.628586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bf394-0d4d-45e0-ad1d-ebfaf75c70fd None admin] RBAC: Authorizing `identity:create_grant(project_id=807a635f1d50461091b40698766b4912, user_id=09dc8be2f93b42bbbddb6b46e1c23a12, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:19.628586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bf394-0d4d-45e0-ad1d-ebfaf75c70fd None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:19.643947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 121/257] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:19 2022] PUT /identity/v3/projects/807a635f1d50461091b40698766b4912/users/09dc8be2f93b42bbbddb6b46e1c23a12/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:19.645149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d571db4-3ca1-4d6f-9aa1-993e31914e82 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.645604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d571db4-3ca1-4d6f-9aa1-993e31914e82 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.645988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d571db4-3ca1-4d6f-9aa1-993e31914e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.674371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3d571db4-3ca1-4d6f-9aa1-993e31914e82 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:19.675765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3d571db4-3ca1-4d6f-9aa1-993e31914e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:19.719257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:19.847277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 122/258] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:19.893693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d571db4-3ca1-4d6f-9aa1-993e31914e82 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:19.894575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a53e2-cead-4c79-a005-e2491359dc08 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:19.942286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:19.960088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:19.963622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:19.964315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:19.964797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:19.970278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:19.970785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:19.972395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:19.985841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 137/259] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:19.991915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:20.102214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d571db4-3ca1-4d6f-9aa1-993e31914e82 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:20.155272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:20.156025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:20.156308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:20.156606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:20.176424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:20.192262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:20.193209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:20.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:20.198478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 123/260] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:20.206735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:20.236932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:20.247306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc455f47-aa38-40c8-b861-5a0c873096b4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:20.368512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c453ad-2ba0-4e64-a894-ad19165f0daf None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:20.381891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:20.382788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:20.383067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:20.383341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] PATH_INFO: `/v3/users/9322aac686b74502b989a426d4dd86c4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:20.384550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] RBAC: Authorizing `identity:get_user(user_id=9322aac686b74502b989a426d4dd86c4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:20.386569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:20.388505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 138/261] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:19 2022] GET /identity/v3/users/9322aac686b74502b989a426d4dd86c4 => generated 283 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:20.397550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:20.407950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:20.408696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:20.408984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:20.409264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:20.410031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:20.410343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:20.411735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:20.421577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 124/262] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:20.427505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:20.532514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:20.548196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44457a26-be45-46d1-b5b4-e28dcf821798 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:20.591387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:20.592264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:20.592542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:20.592839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] PATH_INFO: `/v3/users/9322aac686b74502b989a426d4dd86c4` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:20.594161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] RBAC: Authorizing `identity:get_user(user_id=9322aac686b74502b989a426d4dd86c4)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:20.595629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:20.599488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 125/263] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:20 2022] GET /identity/v3/users/9322aac686b74502b989a426d4dd86c4 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:20.600055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:20.607341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:20.661751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56087bf-62a5-4d67-84f4-4db6f26bddfb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:20.709062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:20.710115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:20.710602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:20.711099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:20.784544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ffb53-90c7-4a06-8947-8e545d7c51ef None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:20.834721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:20.836333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:20.837169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:20.837757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:20.854381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:20.855898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:20.860097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:20.915172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:20.962298 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 139/264] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:20.963151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:20.964735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:20.967687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:21.136523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce1fe4-5301-4625-bc56-4021e8425dd2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:21.170806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 126/265] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:05:21.176798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:21.203574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:21.204449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.204776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.205057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.206382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:21.207846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:21.209783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 140/266] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:20 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:21.216360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:21.256817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:21.262757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:21.360570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67a14a-2197-48ca-93ca-eaeebb8c9baa None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:21.386756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304dfd-67bd-4754-82be-d9f82896d3ec None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:21.406880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:21.408000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.408461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.408923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.412193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:21.413846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:21.418297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 127/267] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:21 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:21.424152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:21.475491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d1fed-28a1-43b8-88db-c52464bed20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:21.476439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1fed-28a1-43b8-88db-c52464bed20f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.477018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1fed-28a1-43b8-88db-c52464bed20f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.477446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1fed-28a1-43b8-88db-c52464bed20f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.478426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d1fed-28a1-43b8-88db-c52464bed20f None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=807a635f1d50461091b40698766b4912, user.id=9322aac686b74502b989a426d4dd86c4) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:21.479497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d1fed-28a1-43b8-88db-c52464bed20f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:21.481182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d1fed-28a1-43b8-88db-c52464bed20f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:21.489233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 141/268] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:21 2022] GET /identity/v3/role_assignments?user.id=9322aac686b74502b989a426d4dd86c4&scope.project.id=807a635f1d50461091b40698766b4912&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:21.619483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c0db4-2829-4abb-9dd2-10263d70f2b5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:21.637235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:21.670883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ecf0c-bc55-4101-b321-3b4b28f8a3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:21.671630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ecf0c-bc55-4101-b321-3b4b28f8a3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.671938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ecf0c-bc55-4101-b321-3b4b28f8a3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.672225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ecf0c-bc55-4101-b321-3b4b28f8a3e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.673191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ecf0c-bc55-4101-b321-3b4b28f8a3e5 None admin] RBAC: Adding query filter params (role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=807a635f1d50461091b40698766b4912, user.id=9322aac686b74502b989a426d4dd86c4) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:21.673539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ecf0c-bc55-4101-b321-3b4b28f8a3e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:21.675051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ecf0c-bc55-4101-b321-3b4b28f8a3e5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:21.686080 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 128/269] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:21 2022] GET /identity/v3/role_assignments?user.id=9322aac686b74502b989a426d4dd86c4&scope.project.id=807a635f1d50461091b40698766b4912&role.id=cc730cc298404131a99c3547377a5e49 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:21.750956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82703ce-e43d-4def-bdca-effcb95f17b1 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.751482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82703ce-e43d-4def-bdca-effcb95f17b1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.751587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82703ce-e43d-4def-bdca-effcb95f17b1 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.753332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 142/270] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:21.759488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b197-55a7-4401-a013-b042f06d0559 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.759810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b197-55a7-4401-a013-b042f06d0559 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.760643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7b197-55a7-4401-a013-b042f06d0559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.787514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-25e7b197-55a7-4401-a013-b042f06d0559 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:21.788908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-25e7b197-55a7-4401-a013-b042f06d0559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:21.888162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 129/271] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:21.930126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82703ce-e43d-4def-bdca-effcb95f17b1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:21.931725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d4f5-fc3a-43ed-b68f-84423775891f None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.932060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d4f5-fc3a-43ed-b68f-84423775891f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.932624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d4f5-fc3a-43ed-b68f-84423775891f None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.934018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 130/272] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:21.941081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec85dc1-25a2-4dc2-ba73-f25113551ad5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:21.941364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec85dc1-25a2-4dc2-ba73-f25113551ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:21.941635 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec85dc1-25a2-4dc2-ba73-f25113551ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:21.969611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8ec85dc1-25a2-4dc2-ba73-f25113551ad5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:21.970804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8ec85dc1-25a2-4dc2-ba73-f25113551ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:22.071478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 131/273] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:22.079837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:22.096240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec85dc1-25a2-4dc2-ba73-f25113551ad5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:22.098928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82703ce-e43d-4def-bdca-effcb95f17b1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:22.144767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:22.152965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:22.153805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.154093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.154373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.161330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:22.162791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:22.164634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 143/274] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:22.171645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:22.240662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec85dc1-25a2-4dc2-ba73-f25113551ad5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:22.278696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:22.279593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.279874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.280143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.294832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:22.305123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:22.306218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:22.307471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 132/275] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:22.313685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:22.318870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d1719-b5e9-47c9-87ee-385849da1237 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:22.359142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266205c-0600-4889-a8e3-e1ca675c57b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:22.360190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266205c-0600-4889-a8e3-e1ca675c57b2 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.360646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266205c-0600-4889-a8e3-e1ca675c57b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.361060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266205c-0600-4889-a8e3-e1ca675c57b2 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.361938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266205c-0600-4889-a8e3-e1ca675c57b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:22.363520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266205c-0600-4889-a8e3-e1ca675c57b2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:22.390969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 144/276] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:22 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:05:22.436012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:22.441809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea45c34-e102-4ee6-91d4-5e13a85d9259 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.442323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea45c34-e102-4ee6-91d4-5e13a85d9259 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.442540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eac1f8-5546-48ef-a787-a76308de622d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:22.442958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea45c34-e102-4ee6-91d4-5e13a85d9259 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.445698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 145/277] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:22.453385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce733f7-3a93-43e8-889a-672f49af6153 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.453673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce733f7-3a93-43e8-889a-672f49af6153 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.453953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce733f7-3a93-43e8-889a-672f49af6153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.476672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9ce733f7-3a93-43e8-889a-672f49af6153 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:22.478483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9ce733f7-3a93-43e8-889a-672f49af6153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:22.486874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:22.487895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.488359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.489166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:22.489843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.491026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:22.491548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:22.493278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:22.505737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 133/278] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:22.511894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:22.573531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 146/279] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:22.604034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce733f7-3a93-43e8-889a-672f49af6153 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:22.651782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c9de1-98ee-4839-a8fa-8b031c07b0b2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:22.696776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650bc98-d8c4-4844-97f8-53093ba38f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:22.698536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650bc98-d8c4-4844-97f8-53093ba38f57 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.699018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650bc98-d8c4-4844-97f8-53093ba38f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.699514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650bc98-d8c4-4844-97f8-53093ba38f57 None admin] PATH_INFO: `/v3/users/a484023724724048ac3c8797b0a3b08a` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.715896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650bc98-d8c4-4844-97f8-53093ba38f57 None admin] RBAC: Authorizing `identity:get_user(user_id=a484023724724048ac3c8797b0a3b08a)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:22.717398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650bc98-d8c4-4844-97f8-53093ba38f57 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:22.719137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 134/280] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:22 2022] GET /identity/v3/users/a484023724724048ac3c8797b0a3b08a => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:22.720439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca796f6-cb57-4b17-8fd7-ae73cad60916 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.720858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca796f6-cb57-4b17-8fd7-ae73cad60916 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.721104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca796f6-cb57-4b17-8fd7-ae73cad60916 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.722670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 135/281] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:22.725970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca796f6-cb57-4b17-8fd7-ae73cad60916 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:22.737984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce733f7-3a93-43e8-889a-672f49af6153 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:22.751918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:22.784635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d22174-01ad-498b-a647-e4e475f4ae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:22.786317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22174-01ad-498b-a647-e4e475f4ae1f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.786897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22174-01ad-498b-a647-e4e475f4ae1f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.787401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22174-01ad-498b-a647-e4e475f4ae1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.799919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d22174-01ad-498b-a647-e4e475f4ae1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e4d22174-01ad-498b-a647-e4e475f4ae1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:22.809149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:22.810038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:22.810830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:22.813812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 147/282] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:22.820271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53aae-7c94-4691-ae08-b41ef14c1cda None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.820927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53aae-7c94-4691-ae08-b41ef14c1cda None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.821510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53aae-7c94-4691-ae08-b41ef14c1cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.828128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:22.844767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b8a53aae-7c94-4691-ae08-b41ef14c1cda None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:22.846573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b8a53aae-7c94-4691-ae08-b41ef14c1cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:22.847142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca796f6-cb57-4b17-8fd7-ae73cad60916 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:22.882939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:22.883598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:22.883880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:22.884153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:22.885600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, user.id=a484023724724048ac3c8797b0a3b08a, scope.system=all) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:22.885905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:22.887332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:22.892924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 136/283] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Jun 24 09:05:22 2022] GET /identity/v3/role_assignments?user.id=a484023724724048ac3c8797b0a3b08a&scope.system=all&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:22.893728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:22.936550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 148/284] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:22.960789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53aae-7c94-4691-ae08-b41ef14c1cda None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:23.014277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:23.047581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e39286-151f-4275-b720-4edf3bcae685 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.091973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:23.093140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:23.093622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:23.094118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:23.095095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:23.095869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:23.097492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:23.107560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 137/285] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:23.115166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:23.214762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53aae-7c94-4691-ae08-b41ef14c1cda None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.219059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c921-392d-4c55-884b-9994512127ee None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.258207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:23.260485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:23.260986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:23.261270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] PATH_INFO: `/v3/users/fb54706bdfaf49e8b3e13fbe7177b602` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:23.263669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] RBAC: Authorizing `identity:get_user(user_id=fb54706bdfaf49e8b3e13fbe7177b602)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:23.265206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:23.275504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 138/286] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:23 2022] GET /identity/v3/users/fb54706bdfaf49e8b3e13fbe7177b602 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:23.278066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:23.298640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:23.299327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:23.308450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:23.309282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:23.309602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:23.309889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:23.333077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:23.341443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:23.342324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:23.343093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 149/287] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:23.356702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:23.496260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765666e5-0938-46fd-8410-089f4ecd8453 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.513362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6089e-908b-4bb8-8c52-1c0cdccfdb3d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.566040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:23.567007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:23.567367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:23.567691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:23.568152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:23.568995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:23.570437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:23.572135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 139/288] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:23 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:23.572780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:23.573071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:23.573357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:23.574129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:23.574439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:23.575807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:23.577640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:23.598438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 150/289] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:23.603018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:23.680905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:23.696942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:23.724640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4913b6-8112-4cc5-a155-49fe76245965 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.753454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb632d-e1a7-4a69-b7c4-e442c34b1bc7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.793207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:23.795181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:23.796158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:23.796994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:23.798221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:23.798746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] PATH_INFO: `/v3/users/09dc8be2f93b42bbbddb6b46e1c23a12` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:23.800007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:23.800971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] RBAC: Authorizing `identity:get_user(user_id=09dc8be2f93b42bbbddb6b46e1c23a12)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:23.801513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912/users/fb54706bdfaf49e8b3e13fbe7177b602/roles/f5a4836824db455d937bdc03a76f1c76` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:23.804151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:23.808401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] RBAC: Authorizing `identity:create_grant(project_id=807a635f1d50461091b40698766b4912, user_id=fb54706bdfaf49e8b3e13fbe7177b602, role_id=f5a4836824db455d937bdc03a76f1c76)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:23.810271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:23.810795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 151/290] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:23 2022] GET /identity/v3/users/09dc8be2f93b42bbbddb6b46e1c23a12 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:23.814293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:23.844622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 140/291] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:05:23 2022] PUT /identity/v3/projects/807a635f1d50461091b40698766b4912/users/fb54706bdfaf49e8b3e13fbe7177b602/roles/f5a4836824db455d937bdc03a76f1c76 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:05:23.954924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e5c89-c10b-498d-a55e-5088db346743 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:23.990639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:24.005190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:24.005939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:24.006229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:24.006510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:24.092912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:24.094682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:24.225198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 152/292] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:24.232517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:24.375559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a64f4-4a4c-4693-b8f8-d7b3d7b9f06f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:24.421127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3da73-a7bb-4e27-8420-ee06ece1928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:24.422308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3da73-a7bb-4e27-8420-ee06ece1928e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:24.422733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3da73-a7bb-4e27-8420-ee06ece1928e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:24.423200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3da73-a7bb-4e27-8420-ee06ece1928e None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:24.426747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3da73-a7bb-4e27-8420-ee06ece1928e None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:24.428401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3da73-a7bb-4e27-8420-ee06ece1928e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:24.430455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 141/293] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:24 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:24.437072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:24.461783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:24.558334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c15c27-069a-4de0-926e-c9dd0245ee10 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:24.601918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591d372-0eb9-4ce9-9581-cb128baad5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:24.602813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591d372-0eb9-4ce9-9581-cb128baad5cb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:24.603293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591d372-0eb9-4ce9-9581-cb128baad5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:24.604346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591d372-0eb9-4ce9-9581-cb128baad5cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:24.605367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591d372-0eb9-4ce9-9581-cb128baad5cb None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=807a635f1d50461091b40698766b4912, user.id=09dc8be2f93b42bbbddb6b46e1c23a12) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:24.605833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591d372-0eb9-4ce9-9581-cb128baad5cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:24.607394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591d372-0eb9-4ce9-9581-cb128baad5cb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:24.614162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 153/294] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:24 2022] GET /identity/v3/role_assignments?user.id=09dc8be2f93b42bbbddb6b46e1c23a12&scope.project.id=807a635f1d50461091b40698766b4912&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:24.976139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c65817-e718-4119-a1d5-1eb9e43fc1de None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:24.977199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c65817-e718-4119-a1d5-1eb9e43fc1de None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:24.977681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c65817-e718-4119-a1d5-1eb9e43fc1de None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:24.979749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 142/295] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:24.983748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094759d2-0374-445c-b76b-4b03f71195e7 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:24.984705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094759d2-0374-445c-b76b-4b03f71195e7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:24.984904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4fcd0-0da1-435b-8132-873febe2a8a9 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:24.985150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4fcd0-0da1-435b-8132-873febe2a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:24.985432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4fcd0-0da1-435b-8132-873febe2a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:24.985779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094759d2-0374-445c-b76b-4b03f71195e7 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:24.987727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 154/296] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:24.997986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181a17-d3f2-49ea-8982-f4fde47612d4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:24.998581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181a17-d3f2-49ea-8982-f4fde47612d4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:24.999060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f181a17-d3f2-49ea-8982-f4fde47612d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.003949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:25.013752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3ce4fcd0-0da1-435b-8132-873febe2a8a9 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:25.017510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3ce4fcd0-0da1-435b-8132-873febe2a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:25.028465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4f181a17-d3f2-49ea-8982-f4fde47612d4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:25.028465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4f181a17-d3f2-49ea-8982-f4fde47612d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:25.110156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 143/297] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:25.112807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 155/298] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:25.133787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4fcd0-0da1-435b-8132-873febe2a8a9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:25.137474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f181a17-d3f2-49ea-8982-f4fde47612d4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:25.156825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:25.241597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f181a17-d3f2-49ea-8982-f4fde47612d4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:25.242579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4fcd0-0da1-435b-8132-873febe2a8a9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:25.278902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:25.279701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.279978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.280253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.281478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:25.281655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:25.282567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.282851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.282950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:25.283127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.284749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 156/299] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:25.286074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:25.290474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:25.297840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:25.308329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:25.309308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:25.310069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:25.310657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 144/300] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:25.315742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:25.408433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:25.429718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884f8e5-a0bb-446e-88d5-196f268c90ce None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:25.455348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e2135-6e85-43ca-bf3d-740bd2a8138a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:25.472281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:25.472984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.473412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.473705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] PATH_INFO: `/v3/groups` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.474397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:25.476497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:25.498209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 157/301] 10.222.0.52 () {60 vars in 1134 bytes} [Fri Jun 24 09:05:25 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:25.506903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8c5c9-5aff-4d74-8b49-8feb31d3f031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:25.508132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c5c9-5aff-4d74-8b49-8feb31d3f031 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.508719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c5c9-5aff-4d74-8b49-8feb31d3f031 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.509243 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c5c9-5aff-4d74-8b49-8feb31d3f031 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.510819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8c5c9-5aff-4d74-8b49-8feb31d3f031 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:25.511329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8c5c9-5aff-4d74-8b49-8feb31d3f031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:25.513238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8c5c9-5aff-4d74-8b49-8feb31d3f031 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:25.524464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 145/302] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:25.534044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:25.655974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc8a4-d068-4f0d-8550-2cafc08b3d49 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:25.689452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285cc81-5ccc-47a7-bf97-74ff916e067a None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.689744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285cc81-5ccc-47a7-bf97-74ff916e067a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.690093 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285cc81-5ccc-47a7-bf97-74ff916e067a None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.691621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 146/303] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:25.696169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:25.696692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbd9bc-47bc-4080-800b-418493b96563 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.696978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbd9bc-47bc-4080-800b-418493b96563 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.697153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.697241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbd9bc-47bc-4080-800b-418493b96563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.697473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.697665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] PATH_INFO: `/v3/users/a45d81908cce49708a9d8b7740b319ca` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.713126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] RBAC: Authorizing `identity:get_user(user_id=a45d81908cce49708a9d8b7740b319ca)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:25.714609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:25.716520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 158/304] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:25 2022] GET /identity/v3/users/a45d81908cce49708a9d8b7740b319ca => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:25.716674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-37dbd9bc-47bc-4080-800b-418493b96563 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:25.717676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-37dbd9bc-47bc-4080-800b-418493b96563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:25.722506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:25.725491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:25.795548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:25.813714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 147/305] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:25.835820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbd9bc-47bc-4080-800b-418493b96563 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:25.868434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb3796-5981-41b7-8c61-90a3fc2d65ed None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:25.917200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:25.917866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.918147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.918424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:25.919258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] RBAC: Adding query filter params (role.id=f5a4836824db455d937bdc03a76f1c76, user.id=a45d81908cce49708a9d8b7740b319ca, scope.system=all) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:25.919626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:25.921915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:25.930014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 159/306] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Jun 24 09:05:25 2022] GET /identity/v3/role_assignments?user.id=a45d81908cce49708a9d8b7740b319ca&scope.system=all&role.id=f5a4836824db455d937bdc03a76f1c76 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:25.952999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbd9bc-47bc-4080-800b-418493b96563 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:25.996095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:25.996946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:25.997233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:25.997514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.010240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:26.018390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:26.022177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:26.022956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 148/307] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:26.026550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:26.110910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:26.153004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63f26-018a-4fb2-a81c-c25f683e4847 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:26.191726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05560c53-4995-4b9b-8713-de00599ba23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:26.192449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05560c53-4995-4b9b-8713-de00599ba23b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:26.192755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05560c53-4995-4b9b-8713-de00599ba23b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:26.193072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05560c53-4995-4b9b-8713-de00599ba23b None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.193904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05560c53-4995-4b9b-8713-de00599ba23b None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:26.194208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05560c53-4995-4b9b-8713-de00599ba23b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:26.195740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05560c53-4995-4b9b-8713-de00599ba23b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:26.205020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 160/308] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:26.210524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:26.266176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:26.307977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b884-17ec-4ade-8814-aa7f8c3b1f1f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:26.313960 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b884-17ec-4ade-8814-aa7f8c3b1f1f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:26.315899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b884-17ec-4ade-8814-aa7f8c3b1f1f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.321673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 161/309] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:26.324157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3d89-fe6f-4929-9750-e30c482ac777 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:26.324445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3d89-fe6f-4929-9750-e30c482ac777 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:26.324752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3d89-fe6f-4929-9750-e30c482ac777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.342066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4d3e3-981a-469f-986f-a9f95d9bff0b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:26.352404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-99ea3d89-fe6f-4929-9750-e30c482ac777 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:26.353666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-99ea3d89-fe6f-4929-9750-e30c482ac777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:26.402936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:26.403880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:26.404171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:26.404443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] PATH_INFO: `/v3/users/a484023724724048ac3c8797b0a3b08a` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.405694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] RBAC: Authorizing `identity:get_user(user_id=a484023724724048ac3c8797b0a3b08a)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:26.407142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:26.408815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 149/310] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:26 2022] GET /identity/v3/users/a484023724724048ac3c8797b0a3b08a => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:26.414532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:26.439209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 162/311] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:26.463754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea3d89-fe6f-4929-9750-e30c482ac777 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:26.526586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:26.547924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628e5c2-088f-4fb7-aacd-b0a560fc30e7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:26.578107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:26.584245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea3d89-fe6f-4929-9750-e30c482ac777 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:26.600508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:26.601407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:26.601689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:26.601971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] PATH_INFO: `/v3/system/users/a484023724724048ac3c8797b0a3b08a/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.603444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a484023724724048ac3c8797b0a3b08a, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:26.605003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:26.622386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 150/312] 10.222.0.52 () {58 vars in 1334 bytes} [Fri Jun 24 09:05:26 2022] PUT /identity/v3/system/users/a484023724724048ac3c8797b0a3b08a/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:05:26.639013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:26.640263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:26.640878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:26.641446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.655280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:26.664325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:26.665325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:26.666199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:26.668881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 163/313] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:26.676137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:26.809994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a9a92-fd93-4f67-afc2-c6c016b88c52 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:26.848294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:26.849004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:26.849291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:26.849566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:26.850322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:26.850631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:26.852016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:26.861249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 151/314] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:26.866849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:26.991210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf564a-d9d2-409c-88e2-23a12069246e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:27.009800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:27.030931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:27.032617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:27.033139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:27.033622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] PATH_INFO: `/v3/users/fb54706bdfaf49e8b3e13fbe7177b602` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:27.035196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] RBAC: Authorizing `identity:get_user(user_id=fb54706bdfaf49e8b3e13fbe7177b602)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:27.040644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:27.043763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 164/315] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:26 2022] GET /identity/v3/users/fb54706bdfaf49e8b3e13fbe7177b602 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:27.045166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:27.092927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:27.155939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56bc85-894c-4f26-80e6-52b4c216252e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:27.192335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:27.193031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:27.193319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:27.193604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:27.265630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:27.267312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:27.322179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:27.384337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 152/316] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:27.385778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:27.492344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679c97e-7000-44aa-83ef-e4062c7e46b7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:27.529500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9951df-4847-460a-b3a0-5d39a95bcaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:27.530566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9951df-4847-460a-b3a0-5d39a95bcaee None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:27.530852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9951df-4847-460a-b3a0-5d39a95bcaee None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:27.531132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9951df-4847-460a-b3a0-5d39a95bcaee None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:27.532332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9951df-4847-460a-b3a0-5d39a95bcaee None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:27.533815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9951df-4847-460a-b3a0-5d39a95bcaee None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:27.535844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 165/317] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:27 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:27.542557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:27.646393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1286a-8638-4b97-89c0-0af7ced6784b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:27.684798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a2251-8cfd-4e02-9ac1-2bf583e88832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:27.685452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a2251-8cfd-4e02-9ac1-2bf583e88832 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:27.685788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a2251-8cfd-4e02-9ac1-2bf583e88832 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:27.686066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a2251-8cfd-4e02-9ac1-2bf583e88832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:27.686938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a2251-8cfd-4e02-9ac1-2bf583e88832 None admin] RBAC: Adding query filter params (role.id=f5a4836824db455d937bdc03a76f1c76, scope.project.id=807a635f1d50461091b40698766b4912, user.id=fb54706bdfaf49e8b3e13fbe7177b602) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:27.687249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a2251-8cfd-4e02-9ac1-2bf583e88832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:27.688773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a2251-8cfd-4e02-9ac1-2bf583e88832 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:27.700724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 153/318] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:27 2022] GET /identity/v3/role_assignments?user.id=fb54706bdfaf49e8b3e13fbe7177b602&scope.project.id=807a635f1d50461091b40698766b4912&role.id=f5a4836824db455d937bdc03a76f1c76 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:27.773656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8a22d-614c-4a6f-8d0d-050c52840fc5 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:27.774009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8a22d-614c-4a6f-8d0d-050c52840fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:27.774301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8a22d-614c-4a6f-8d0d-050c52840fc5 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:27.777137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 166/319] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:27.782883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123528f-de1a-4a2b-88f5-69c86ea4db05 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:27.783158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123528f-de1a-4a2b-88f5-69c86ea4db05 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:27.783351 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123528f-de1a-4a2b-88f5-69c86ea4db05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:27.794935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:27.801075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7123528f-de1a-4a2b-88f5-69c86ea4db05 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:27.802274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7123528f-de1a-4a2b-88f5-69c86ea4db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:27.877984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 154/320] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:27.902765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8a22d-614c-4a6f-8d0d-050c52840fc5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:27.978540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:28.029447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8a22d-614c-4a6f-8d0d-050c52840fc5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:28.068173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a4232-3a87-420c-8b43-4bd1c6fe0d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:28.068997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a4232-3a87-420c-8b43-4bd1c6fe0d1d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.069281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a4232-3a87-420c-8b43-4bd1c6fe0d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.069558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a4232-3a87-420c-8b43-4bd1c6fe0d1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.070721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a4232-3a87-420c-8b43-4bd1c6fe0d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:28.072164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a4232-3a87-420c-8b43-4bd1c6fe0d1d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:28.074530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 167/321] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:28.079929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123528f-de1a-4a2b-88f5-69c86ea4db05 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:28.083188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03aba35-41fc-47e9-b0e4-9f1d9b3261ad None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.083484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03aba35-41fc-47e9-b0e4-9f1d9b3261ad None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.083770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03aba35-41fc-47e9-b0e4-9f1d9b3261ad None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.085235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 168/322] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:28.090706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1f8e1-d62e-4814-a672-fc124bc932fa None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.090997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1f8e1-d62e-4814-a672-fc124bc932fa None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.091280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1f8e1-d62e-4814-a672-fc124bc932fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.109811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-21c1f8e1-d62e-4814-a672-fc124bc932fa None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:28.110992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-21c1f8e1-d62e-4814-a672-fc124bc932fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:28.192709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123528f-de1a-4a2b-88f5-69c86ea4db05 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:28.194599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 169/323] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:28.218116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1f8e1-d62e-4814-a672-fc124bc932fa None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:28.230964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:28.231644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.232267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] PATH_INFO: `/v3/groups` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.232919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:28.234412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:28.239402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:28.261192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 155/324] 10.222.0.52 () {60 vars in 1134 bytes} [Fri Jun 24 09:05:28 2022] POST /identity/v3/groups => generated 226 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:28.286795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:28.344905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1f8e1-d62e-4814-a672-fc124bc932fa None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:28.382721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:28.383545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.383829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.384105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.396307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:28.404191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:28.405157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:28.405932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 170/325] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:28.411596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:28.521316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f339d9-33ea-4463-b592-5a2db55ccd89 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:28.560970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:28.561653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.561942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.562222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.562990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:28.563300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:28.564724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:28.574000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 156/326] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:28.579327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:28.695397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34dac8-1cd1-4651-b57a-2ac7f73d7eb1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:28.713907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:28.743210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3671b8-30cb-4143-ad41-3e84899f5641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:28.744142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3671b8-30cb-4143-ad41-3e84899f5641 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.745011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3671b8-30cb-4143-ad41-3e84899f5641 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.745291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3671b8-30cb-4143-ad41-3e84899f5641 None admin] PATH_INFO: `/v3/users/a45d81908cce49708a9d8b7740b319ca` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.746526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3671b8-30cb-4143-ad41-3e84899f5641 None admin] RBAC: Authorizing `identity:get_user(user_id=a45d81908cce49708a9d8b7740b319ca)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:28.747991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3671b8-30cb-4143-ad41-3e84899f5641 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:28.749967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 171/327] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:28 2022] GET /identity/v3/users/a45d81908cce49708a9d8b7740b319ca => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:28.755572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:28.824089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:28.870963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538d840-aca2-456a-9041-2f660ee7d712 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:28.873060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f264b0-a9fe-488e-8f74-cd27b390027e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.873579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f264b0-a9fe-488e-8f74-cd27b390027e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.874051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f264b0-a9fe-488e-8f74-cd27b390027e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.877139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 172/328] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:28.884836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366371-16b9-459c-9e86-5c3619e6de67 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.885403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366371-16b9-459c-9e86-5c3619e6de67 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.885861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366371-16b9-459c-9e86-5c3619e6de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.911221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f7366371-16b9-459c-9e86-5c3619e6de67 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:28.911404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f7366371-16b9-459c-9e86-5c3619e6de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:28.915339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:28.917137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:28.917617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:28.918055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] PATH_INFO: `/v3/system/users/a45d81908cce49708a9d8b7740b319ca/roles/f5a4836824db455d937bdc03a76f1c76` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:28.920463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a45d81908cce49708a9d8b7740b319ca, role_id=f5a4836824db455d937bdc03a76f1c76)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:28.922065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:28.939397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 157/329] 10.222.0.52 () {58 vars in 1334 bytes} [Fri Jun 24 09:05:28 2022] PUT /identity/v3/system/users/a45d81908cce49708a9d8b7740b319ca/roles/f5a4836824db455d937bdc03a76f1c76 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:29.010122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 173/330] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:29.036995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:29.163574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729c67f-b6e7-49f1-b058-442c7a8b6bb9 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:29.190887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:29.203880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:29.205198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:29.205622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:29.206053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:29.221693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:29.230741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:29.231930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:29.234631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:29.237368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 158/331] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:29.244588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7366371-16b9-459c-9e86-5c3619e6de67 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:29.382632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7366371-16b9-459c-9e86-5c3619e6de67 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:29.413998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:29.439626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:29.440313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:29.440650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:29.440933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:29.441696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:29.442004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:29.443376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:29.456713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 174/332] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:29.462181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:29.597235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564b6a6-368f-46d0-a4dc-63ae3f630257 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:29.643306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9e4cc-9f85-4e6e-8446-2b84b9bc52f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:29.644122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e4cc-9f85-4e6e-8446-2b84b9bc52f9 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:29.644405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e4cc-9f85-4e6e-8446-2b84b9bc52f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:29.644720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e4cc-9f85-4e6e-8446-2b84b9bc52f9 None admin] PATH_INFO: `/v3/users/a484023724724048ac3c8797b0a3b08a` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:29.645942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9e4cc-9f85-4e6e-8446-2b84b9bc52f9 None admin] RBAC: Authorizing `identity:get_user(user_id=a484023724724048ac3c8797b0a3b08a)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:29.647386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9e4cc-9f85-4e6e-8446-2b84b9bc52f9 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:29.649418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 159/333] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:29 2022] GET /identity/v3/users/a484023724724048ac3c8797b0a3b08a => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:29.654818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:29.803938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d831d18-5707-434a-8d4a-3442a60d9c9c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:29.845274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49756ea-baf9-4f44-a861-2ea039dedb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:29.845983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49756ea-baf9-4f44-a861-2ea039dedb32 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:29.846270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49756ea-baf9-4f44-a861-2ea039dedb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:29.846553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49756ea-baf9-4f44-a861-2ea039dedb32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:29.847389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49756ea-baf9-4f44-a861-2ea039dedb32 None admin] RBAC: Adding query filter params (role.id=0fc9d0c176a542f78847c87ede7b7b17, user.id=a484023724724048ac3c8797b0a3b08a, scope.system=all) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:29.847699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49756ea-baf9-4f44-a861-2ea039dedb32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:29.849192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49756ea-baf9-4f44-a861-2ea039dedb32 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:29.863936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 175/334] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Jun 24 09:05:29 2022] GET /identity/v3/role_assignments?user.id=a484023724724048ac3c8797b0a3b08a&scope.system=all&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:31.043110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac268a-705d-4c12-aba7-94256cec57cd None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:31.043613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac268a-705d-4c12-aba7-94256cec57cd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:31.043731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac268a-705d-4c12-aba7-94256cec57cd None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:31.045335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 160/335] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:31.062938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12c67e-b98c-4441-80c6-c8bca80e2346 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:31.063436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12c67e-b98c-4441-80c6-c8bca80e2346 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:31.063509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12c67e-b98c-4441-80c6-c8bca80e2346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:31.089581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0d12c67e-b98c-4441-80c6-c8bca80e2346 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:31.090782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0d12c67e-b98c-4441-80c6-c8bca80e2346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:31.118822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:31.174330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 176/336] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:31.211895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac268a-705d-4c12-aba7-94256cec57cd None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:31.280898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:31.344523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac268a-705d-4c12-aba7-94256cec57cd None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:31.381605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:31.382410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:31.382692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:31.382967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:31.397063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:31.405107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:31.406010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:31.406800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 161/337] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:31.412014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12c67e-b98c-4441-80c6-c8bca80e2346 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:31.533221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12c67e-b98c-4441-80c6-c8bca80e2346 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:31.579098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632be25-3fc0-4dbd-9b51-2d1a32bb7178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:31.579774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632be25-3fc0-4dbd-9b51-2d1a32bb7178 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:31.580057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632be25-3fc0-4dbd-9b51-2d1a32bb7178 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:31.580337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632be25-3fc0-4dbd-9b51-2d1a32bb7178 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:31.581118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632be25-3fc0-4dbd-9b51-2d1a32bb7178 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:31.581422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632be25-3fc0-4dbd-9b51-2d1a32bb7178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:31.582860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632be25-3fc0-4dbd-9b51-2d1a32bb7178 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:31.595141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 177/338] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:31.600660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:31.716735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92527ebc-5bae-48c8-9160-892e458b1a08 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:31.729383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:31.754717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:31.755373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:31.755657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:31.755934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:31.827962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:31.829503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:31.937229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c40aa7-9e06-41b7-aa85-4285543965a8 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:31.937525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c40aa7-9e06-41b7-aa85-4285543965a8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:31.937705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c40aa7-9e06-41b7-aa85-4285543965a8 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:31.939318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 178/339] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:31.944604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477f65e-8ce3-474f-ba95-57b1c4502f79 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:31.944892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477f65e-8ce3-474f-ba95-57b1c4502f79 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:31.945176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477f65e-8ce3-474f-ba95-57b1c4502f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:31.971422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 162/340] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:31.974055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3477f65e-8ce3-474f-ba95-57b1c4502f79 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:31.975254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3477f65e-8ce3-474f-ba95-57b1c4502f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:31.976902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:32.004339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:32.022199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:32.125194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 179/341] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:32.161028 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477f65e-8ce3-474f-ba95-57b1c4502f79 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:32.170265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8334-758b-4c4d-8510-533a6f0df80f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:32.230120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389bb85-1801-445d-b6a7-82ff402ead95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:32.231466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bb85-1801-445d-b6a7-82ff402ead95 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.231956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bb85-1801-445d-b6a7-82ff402ead95 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.232424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bb85-1801-445d-b6a7-82ff402ead95 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.233916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389bb85-1801-445d-b6a7-82ff402ead95 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:32.235676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389bb85-1801-445d-b6a7-82ff402ead95 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:32.238142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 163/342] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:31 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:32.243706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d282f7-4111-4d21-8355-31725c55cf4c None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.243996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d282f7-4111-4d21-8355-31725c55cf4c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.245135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d282f7-4111-4d21-8355-31725c55cf4c None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.247259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 164/343] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:32.248000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d282f7-4111-4d21-8355-31725c55cf4c None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:32.359160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477f65e-8ce3-474f-ba95-57b1c4502f79 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:32.401034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3f556-30aa-4d64-8e39-77b4f706f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:32.401840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3f556-30aa-4d64-8e39-77b4f706f63a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.402127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3f556-30aa-4d64-8e39-77b4f706f63a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.402406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3f556-30aa-4d64-8e39-77b4f706f63a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.418812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:32.427086 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3f556-30aa-4d64-8e39-77b4f706f63a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6ea3f556-30aa-4d64-8e39-77b4f706f63a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:32.435745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:32.436775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:32.437672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:32.438343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 180/344] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:32.439603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c809c-d45e-4ae2-8a20-2c204ffc9785 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.445410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c809c-d45e-4ae2-8a20-2c204ffc9785 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.445754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c809c-d45e-4ae2-8a20-2c204ffc9785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.475061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d282f7-4111-4d21-8355-31725c55cf4c None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:32.489793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2d8c809c-d45e-4ae2-8a20-2c204ffc9785 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:32.490931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2d8c809c-d45e-4ae2-8a20-2c204ffc9785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:32.500927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:32.556470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:32.558704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.559873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.560532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.575750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:32.577699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:32.581829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 165/345] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:32 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:32.584044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:32.607909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 181/346] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:32.610855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c809c-d45e-4ae2-8a20-2c204ffc9785 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:32.738707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c5b36-4a62-4117-a9cc-aeb561bb6a1b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:32.756343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c809c-d45e-4ae2-8a20-2c204ffc9785 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:32.780179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:32.780895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.781186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.781493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.782244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:32.782555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:32.783937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:32.797798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 166/347] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:32.798523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:32.803022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:32.805532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc7e1e-6b60-41e2-a030-3bca6a698ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:32.806315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc7e1e-6b60-41e2-a030-3bca6a698ddc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.806607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc7e1e-6b60-41e2-a030-3bca6a698ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.806879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc7e1e-6b60-41e2-a030-3bca6a698ddc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.807984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc7e1e-6b60-41e2-a030-3bca6a698ddc None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:32.808292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc7e1e-6b60-41e2-a030-3bca6a698ddc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:32.809871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc7e1e-6b60-41e2-a030-3bca6a698ddc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:32.817784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 182/348] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:32 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:32.824621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334e46-3422-446c-bd91-27999d998da7 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.824621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334e46-3422-446c-bd91-27999d998da7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.824621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334e46-3422-446c-bd91-27999d998da7 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.824621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 183/349] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:32.824621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841dc29-8289-42bb-9402-3591d691972a None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.824621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841dc29-8289-42bb-9402-3591d691972a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.825139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841dc29-8289-42bb-9402-3591d691972a None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:32.825139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 184/350] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:32.825294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841dc29-8289-42bb-9402-3591d691972a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:32.837377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:32.955729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58763d0-5896-45e5-9435-576d728e1594 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:32.967273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841dc29-8289-42bb-9402-3591d691972a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:32.997745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009dac7-0dc1-4922-85c9-8213b12f4f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:32.998687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009dac7-0dc1-4922-85c9-8213b12f4f6e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:32.998960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009dac7-0dc1-4922-85c9-8213b12f4f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:32.999289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009dac7-0dc1-4922-85c9-8213b12f4f6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.007955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:33.008540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.008853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.009133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.013012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009dac7-0dc1-4922-85c9-8213b12f4f6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-5009dac7-0dc1-4922-85c9-8213b12f4f6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:33.028308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:33.029305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:33.030128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 167/351] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:33.036489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57772807-6063-4c3e-93de-1d2f5c7a9e36 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.038953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57772807-6063-4c3e-93de-1d2f5c7a9e36 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.039295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57772807-6063-4c3e-93de-1d2f5c7a9e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.058459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-57772807-6063-4c3e-93de-1d2f5c7a9e36 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:33.059826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-57772807-6063-4c3e-93de-1d2f5c7a9e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:33.104629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:33.106441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:33.120741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:33.142531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:33.152536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 168/352] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:33.153954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951722d9-a49e-4386-a70d-009ce7fe6cb4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.154192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951722d9-a49e-4386-a70d-009ce7fe6cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.154471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951722d9-a49e-4386-a70d-009ce7fe6cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.174872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-951722d9-a49e-4386-a70d-009ce7fe6cb4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:33.176411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-951722d9-a49e-4386-a70d-009ce7fe6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:33.262907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 185/353] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:33.263671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:33.267882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 169/354] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:33.268727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3675b-b98d-425f-b703-94540c69efdc None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.269017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3675b-b98d-425f-b703-94540c69efdc None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.269296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3675b-b98d-425f-b703-94540c69efdc None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.270594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 170/355] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:33.271291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3675b-b98d-425f-b703-94540c69efdc None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:33.397551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c102541-41f0-476f-8187-b84a9f49b95e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:33.410863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:33.440841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3675b-b98d-425f-b703-94540c69efdc None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:33.450181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:33.451731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.452273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.452573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.453532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:33.453879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:33.455273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:33.467692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 186/356] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:33.469121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:33.474485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:33.497467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f85ea-4620-47af-9d23-8be8810140f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:33.498365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f85ea-4620-47af-9d23-8be8810140f5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.498647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f85ea-4620-47af-9d23-8be8810140f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.498918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f85ea-4620-47af-9d23-8be8810140f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.516043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f85ea-4620-47af-9d23-8be8810140f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-8b4f85ea-4620-47af-9d23-8be8810140f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:33.526472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:33.527468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:05:33.528395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:33.529631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 171/357] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:33.530491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b9d3d-027d-4ca6-8c30-ff950de936e0 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.530765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b9d3d-027d-4ca6-8c30-ff950de936e0 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.531135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b9d3d-027d-4ca6-8c30-ff950de936e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.552437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7b2b9d3d-027d-4ca6-8c30-ff950de936e0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:33.553655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7b2b9d3d-027d-4ca6-8c30-ff950de936e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:33.615096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf56f5-84bd-47c4-b78a-e5905724e8a3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:33.650129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 172/358] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:33.650790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b9d3d-027d-4ca6-8c30-ff950de936e0 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:33.663510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:33.665136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.665687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.666269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.667908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:33.669823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:33.672016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 187/359] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:33 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:33.673048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:33.726132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:33.779194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b9d3d-027d-4ca6-8c30-ff950de936e0 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:33.797798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:33.817519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244920d3-f314-4377-af3f-a25321a9477e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:33.826362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:33.827376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.827679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.827940 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.848833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:33.862026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:33.862967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:33.863839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 173/360] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:33.864535 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:33.872347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:33.873128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:33.873418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:33.873695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:33.963737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:33.965259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.011205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8980-9f2b-4d77-b3f6-24ca6824d61a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.017890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:34.067856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.068736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.069309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.070248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:34.070561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:34.073580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.083510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 174/361] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:34.087140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.122263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 188/362] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:34.127787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.130606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:34.239391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e024b-ce37-49d4-b3db-2e33b6fcfcdd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.264796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18b095-19d3-4e38-b001-3ebc37d10325 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.280747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.281541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.282025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.282138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.296074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:34.303684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:34.304787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:34.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 175/363] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:34.313187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.313395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.314693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.315179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.315528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.317385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:34.319165 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.320932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 189/364] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:34.321776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.327412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:34.435233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:34.443746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3a3e6-534e-42d6-ab0a-cd1cb10bc57f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.451639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b0bb8-be9e-4209-996d-6bc2c547b2a6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.494986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.495210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.495293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.495572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.496332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:34.496902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:34.498297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.500676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.501551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.501829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.502100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] PATH_INFO: `/v3/users/a45d81908cce49708a9d8b7740b319ca` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.503314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] RBAC: Authorizing `identity:get_user(user_id=a45d81908cce49708a9d8b7740b319ca)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:34.504833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.506929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 190/365] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/users/a45d81908cce49708a9d8b7740b319ca => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:34.510184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 176/366] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:34.511013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.511196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.609512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:34.658283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9498f2-0ecb-4de5-b695-9a0ce450115e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.661925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d035d54-4c8b-4afc-b6bf-7e5bcbfb0abd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.702530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.703502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.703856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.704200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.705288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.705550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:34.705956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.706241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.706522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.707271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:34.707358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.707579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:34.711989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.712123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 191/367] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:34.712123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.725096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 177/368] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:34.725947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.750981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:34.851281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516656c8-6327-4b2a-ae48-342f936f6017 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.858609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1a07a-66b7-49e4-bd0f-b7385b26f50c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:34.905449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.906031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.906287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:34.906407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.906577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.907040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:34.907325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:34.907602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:34.908941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=807a635f1d50461091b40698766b4912) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:34.908941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:34.910300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:34.918786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 192/369] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=807a635f1d50461091b40698766b4912&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:34.918786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:34.923901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.016175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.017738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.049070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.072059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388dfcc-d1bd-4023-9c84-a2fb39a80e76 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:35.111981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:35.112751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.113035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.113313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.114141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] RBAC: Adding query filter params (role.id=f5a4836824db455d937bdc03a76f1c76, user.id=a45d81908cce49708a9d8b7740b319ca, scope.system=all) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:35.114452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.115884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.136595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 193/370] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/role_assignments?user.id=a45d81908cce49708a9d8b7740b319ca&scope.system=all&role.id=f5a4836824db455d937bdc03a76f1c76 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:35.136595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.156198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 178/371] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:35.157194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.226802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.274220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bd3f5-baee-4993-898e-5d000c35d42e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:35.283644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf96ae6-f336-406f-9a9c-b6dc3a5ac3d6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:35.302766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.322944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:35.323559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.323848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.324137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.330487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56a25-8ee0-48a4-a46c-75b9771c8e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:35.331367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56a25-8ee0-48a4-a46c-75b9771c8e9e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.331656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56a25-8ee0-48a4-a46c-75b9771c8e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.331937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56a25-8ee0-48a4-a46c-75b9771c8e9e None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.333179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56a25-8ee0-48a4-a46c-75b9771c8e9e None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.334649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56a25-8ee0-48a4-a46c-75b9771c8e9e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.336668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 194/372] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:35.338533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb7ae-884c-4a08-84fe-06e641d74bb8 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.338827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb7ae-884c-4a08-84fe-06e641d74bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.339106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb7ae-884c-4a08-84fe-06e641d74bb8 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.340353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 195/373] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:35.341076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb7ae-884c-4a08-84fe-06e641d74bb8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.409446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.410855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.469661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb7ae-884c-4a08-84fe-06e641d74bb8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:35.501677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.507912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aeba34-bdc9-402b-a02b-3b2784c47fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:35.509017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aeba34-bdc9-402b-a02b-3b2784c47fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.509384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aeba34-bdc9-402b-a02b-3b2784c47fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.510572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aeba34-bdc9-402b-a02b-3b2784c47fd3 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.511814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aeba34-bdc9-402b-a02b-3b2784c47fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.513316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aeba34-bdc9-402b-a02b-3b2784c47fd3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.515212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 196/374] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:35.517825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e73b9-447e-476b-8202-0e168fa34a00 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.518113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e73b9-447e-476b-8202-0e168fa34a00 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.518388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e73b9-447e-476b-8202-0e168fa34a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.523946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 179/375] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:35.524948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.545077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-695e73b9-447e-476b-8202-0e168fa34a00 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:35.546272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-695e73b9-447e-476b-8202-0e168fa34a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:35.549714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.643912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 197/376] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:35.644690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e73b9-447e-476b-8202-0e168fa34a00 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.654376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17b2df-2d6e-49c1-8098-31e3ed389d19 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:35.699049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:35.699649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.699939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.700213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.701128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:35.701444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.702904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.709378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 180/377] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=cc730cc298404131a99c3547377a5e49 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:35.710190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.793569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e73b9-447e-476b-8202-0e168fa34a00 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:35.815745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.836413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b54900-f860-4726-b64a-acbfd4355e43 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:35.836953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:35.847564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:35.849006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.849296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.849573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.854156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.855631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.862457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 198/378] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:35.862457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.902258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:35.903511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:35.903998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:35.904502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] PATH_INFO: `/v3/projects/ccae62deda0a4d329baa9cb863e45e2d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:35.906606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] RBAC: Authorizing `identity:get_project(project_id=ccae62deda0a4d329baa9cb863e45e2d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:35.908774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:35.911856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 181/379] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/projects/ccae62deda0a4d329baa9cb863e45e2d => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:35.912880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:35.992484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9ad28-da07-4e85-9fee-43a81a5eccbf None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:36.040273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:36.041159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:36.041444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:36.041718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:36.047231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0a873-a78d-4f38-a4cf-46d3a3ae6c07 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:36.053074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:36.061508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:36.062405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:36.063612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 199/380] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:36.064832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:36.091198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:36.092204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:36.092818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:36.093305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:36.094406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=807a635f1d50461091b40698766b4912) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:36.094932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:36.096751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:36.103524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 182/381] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:35 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=807a635f1d50461091b40698766b4912&role.id=cc730cc298404131a99c3547377a5e49 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:36.104665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:36.123525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:36.160886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:36.231285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe849-8281-4e8c-a1dc-976e777a7274 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:36.278048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:36.278680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:36.280313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:36.280630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:36.281410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:36.281727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:36.283138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:36.286795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3997f8-bd6e-4500-8f13-fa62484be258 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:36.306655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 183/382] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:36.313488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:36.356844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:36.358005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:36.358293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:36.358571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] PATH_INFO: `/v3/groups/a4acf24e78e6440ca493247d9880f598` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:36.365745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] RBAC: Authorizing `identity:get_group(group_id=a4acf24e78e6440ca493247d9880f598)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:36.367669 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:36.370081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 200/383] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:36 2022] GET /identity/v3/groups/a4acf24e78e6440ca493247d9880f598 => generated 229 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:36.376220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:36.502466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d104f1e-f9a5-410c-b50f-4abfe1b932f7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:36.505396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:36.538001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:36.553067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:36.554299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:36.554665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:36.555016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:36.555931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] RBAC: Adding query filter params (group.id=a4acf24e78e6440ca493247d9880f598, role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=ccae62deda0a4d329baa9cb863e45e2d) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:36.556322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:36.558781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:36.561127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35ecb2-6709-4747-8523-49646df6d932 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:36.569027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 201/384] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:36 2022] GET /identity/v3/role_assignments?group.id=a4acf24e78e6440ca493247d9880f598&scope.project.id=ccae62deda0a4d329baa9cb863e45e2d&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:36.616835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:36.617720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:36.617995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:36.618263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:36.619571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:36.621110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:36.623321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 184/385] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:36 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:36.635336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:36.791968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483292cc-4019-4e36-baef-19396be4f76c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:36.841095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2989e-7e9b-40dd-838c-642230e974f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:36.842003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2989e-7e9b-40dd-838c-642230e974f6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:36.842332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2989e-7e9b-40dd-838c-642230e974f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:36.842965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2989e-7e9b-40dd-838c-642230e974f6 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:36.843802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2989e-7e9b-40dd-838c-642230e974f6 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:36.845279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2989e-7e9b-40dd-838c-642230e974f6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:36.847310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 202/386] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:36 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:36.852574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:36.974887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c666076-cd02-42ad-be22-f2e615cf0e68 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:37.028375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970bf6b-2699-405b-88ac-15373806909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:37.029298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970bf6b-2699-405b-88ac-15373806909d None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:37.029576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970bf6b-2699-405b-88ac-15373806909d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:37.029847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970bf6b-2699-405b-88ac-15373806909d None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/groups/c89dde724c2f41dbbaab789161d80064/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:37.035555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970bf6b-2699-405b-88ac-15373806909d None admin] RBAC: Authorizing `identity:create_grant(project_id=473c53bf5c8740e4aaabb9c9465bd165, group_id=c89dde724c2f41dbbaab789161d80064, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:37.037237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970bf6b-2699-405b-88ac-15373806909d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:37.068239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 185/387] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Jun 24 09:05:36 2022] PUT /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/groups/c89dde724c2f41dbbaab789161d80064/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:37.151771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd64381-61e9-438b-a965-32dc652005fa None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:37.152240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd64381-61e9-438b-a965-32dc652005fa None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:37.152524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd64381-61e9-438b-a965-32dc652005fa None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:37.153995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 203/388] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:37.160902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f82c6f-e7fe-40a6-be0e-e9cbb5149eb0 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:37.161253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f82c6f-e7fe-40a6-be0e-e9cbb5149eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:37.161674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f82c6f-e7fe-40a6-be0e-e9cbb5149eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:37.177799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:37.191782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d3f82c6f-e7fe-40a6-be0e-e9cbb5149eb0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:37.193045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d3f82c6f-e7fe-40a6-be0e-e9cbb5149eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:37.285684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 186/389] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:37.310010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd64381-61e9-438b-a965-32dc652005fa None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:37.408957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:37.479104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd64381-61e9-438b-a965-32dc652005fa None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:37.527242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:37.528041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:37.528320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:37.528620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:37.539704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:37.548056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:37.549017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:37.549806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 204/390] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:37.561177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f82c6f-e7fe-40a6-be0e-e9cbb5149eb0 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:37.708655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f82c6f-e7fe-40a6-be0e-e9cbb5149eb0 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:37.761270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8db472-291b-4e7f-8dcb-c518f174af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:37.761957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8db472-291b-4e7f-8dcb-c518f174af59 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:37.762238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8db472-291b-4e7f-8dcb-c518f174af59 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:37.762511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8db472-291b-4e7f-8dcb-c518f174af59 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:37.763262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8db472-291b-4e7f-8dcb-c518f174af59 None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:37.763561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8db472-291b-4e7f-8dcb-c518f174af59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:37.764960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8db472-291b-4e7f-8dcb-c518f174af59 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:37.767480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:37.774787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 187/391] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:37.781341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:37.884918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecffb2f-aabd-4682-b279-ae5ce879e75d None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:37.885372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecffb2f-aabd-4682-b279-ae5ce879e75d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:37.885828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecffb2f-aabd-4682-b279-ae5ce879e75d None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:37.888675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 188/392] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:37.893290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e359d-abce-40a8-ab01-ef3295358871 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:37.893602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e359d-abce-40a8-ab01-ef3295358871 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:37.893965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e359d-abce-40a8-ab01-ef3295358871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:37.920086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9c9e359d-abce-40a8-ab01-ef3295358871 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:37.921333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9c9e359d-abce-40a8-ab01-ef3295358871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:37.972545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81c8a-5a9e-435d-aa0e-e1361ff5eb0e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:38.008902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:38.043574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:38.044444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:38.044747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:38.045024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:38.047574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:38.049065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:38.050995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 189/393] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:38.051205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 205/394] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:37 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:38.056626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e359d-abce-40a8-ab01-ef3295358871 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:38.094677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:38.218527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e359d-abce-40a8-ab01-ef3295358871 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:38.276640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:38.292975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:38.293922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:38.294214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:38.294483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:38.296880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8953c-e747-4f8b-9658-158ac5fe53bf None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:38.297600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:38.299076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:38.304744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 190/395] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:38 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:38.316353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:38.347495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:38.348448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:38.348762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:38.350032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:38.372575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:38.389674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:38.390587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:38.391390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:38.391976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 206/396] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:38.398343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:38.482537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:38.524014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70342763-8f7d-4a8f-9374-dc18ec9cc56e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:38.603173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:38.604114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:38.604399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:38.606133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912/groups/c89dde724c2f41dbbaab789161d80064/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:38.614587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db7aae-2112-4320-813b-06b38412fbb0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:38.616919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] RBAC: Authorizing `identity:create_grant(project_id=807a635f1d50461091b40698766b4912, group_id=c89dde724c2f41dbbaab789161d80064, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:38.618372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:38.652053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 191/397] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Jun 24 09:05:38 2022] PUT /identity/v3/projects/807a635f1d50461091b40698766b4912/groups/c89dde724c2f41dbbaab789161d80064/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:05:38.696104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448efa37-588f-45ad-87d6-b00bc4ee069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:38.696839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448efa37-588f-45ad-87d6-b00bc4ee069b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:38.697685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448efa37-588f-45ad-87d6-b00bc4ee069b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:38.697974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448efa37-588f-45ad-87d6-b00bc4ee069b None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:38.698746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448efa37-588f-45ad-87d6-b00bc4ee069b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:38.699053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448efa37-588f-45ad-87d6-b00bc4ee069b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:38.700505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448efa37-588f-45ad-87d6-b00bc4ee069b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:38.715545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 207/398] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:38.720924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:38.772550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966e19e-18a5-4b73-92a1-55135f827a2b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:38.772890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966e19e-18a5-4b73-92a1-55135f827a2b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:38.773150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966e19e-18a5-4b73-92a1-55135f827a2b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:38.778448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 208/399] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:38.780387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35f0c-c9ed-4b0a-acbc-95c3e1105eca None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:38.780725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35f0c-c9ed-4b0a-acbc-95c3e1105eca None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:38.781013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35f0c-c9ed-4b0a-acbc-95c3e1105eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:38.800319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a7f35f0c-c9ed-4b0a-acbc-95c3e1105eca None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:38.801579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a7f35f0c-c9ed-4b0a-acbc-95c3e1105eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:38.894775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:38.930793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 209/400] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:38.960147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddab42-92b0-4eb1-b84c-79fb5133f168 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:38.961966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f35f0c-c9ed-4b0a-acbc-95c3e1105eca None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.012012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:39.068617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:39.069946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.070428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.070925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.073397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:39.075109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:39.077025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 192/401] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:38 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:39.082472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.118829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f35f0c-c9ed-4b0a-acbc-95c3e1105eca None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:39.158421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:39.159152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.160193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.160193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.191425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:39.216041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:39.217063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:39.217847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:39.218992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 210/402] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:39.228848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.363869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:39.375344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3755004-c225-4390-a317-6edcf92e3e26 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:39.377730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695cc1de-020a-43da-9d4b-4101ca937af1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:39.401140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:39.413065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:39.413914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.414195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.414521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.415721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:39.419780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:39.419780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 193/403] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:39 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:39.425501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.436157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:39.437319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.438040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.438540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.439499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:39.440031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:39.441744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:39.455509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 211/404] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:39.461803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.562132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9486ad1-636d-4a76-b74c-8dfcb7a7c37f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:39.593356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7782a-ad36-49e8-9b7b-e2150df64c19 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:39.602260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3482d38-d47e-4452-a69d-aca2c3322ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:39.603087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482d38-d47e-4452-a69d-aca2c3322ecc None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.603541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482d38-d47e-4452-a69d-aca2c3322ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.603831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482d38-d47e-4452-a69d-aca2c3322ecc None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/groups/c89dde724c2f41dbbaab789161d80064/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.606673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3482d38-d47e-4452-a69d-aca2c3322ecc None admin] RBAC: Authorizing `identity:create_grant(project_id=473c53bf5c8740e4aaabb9c9465bd165, group_id=c89dde724c2f41dbbaab789161d80064, role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:39.608104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3482d38-d47e-4452-a69d-aca2c3322ecc None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:39.625545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 194/405] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Jun 24 09:05:39 2022] PUT /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165/groups/c89dde724c2f41dbbaab789161d80064/roles/cc730cc298404131a99c3547377a5e49 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:39.628174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011e49-c150-44d2-becc-df63e12fe6c7 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.628504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011e49-c150-44d2-becc-df63e12fe6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.628888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83011e49-c150-44d2-becc-df63e12fe6c7 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.630924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 195/406] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:39.637709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3b36-85ec-440f-8e5a-31fb2c87d6ea None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.638142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3b36-85ec-440f-8e5a-31fb2c87d6ea None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.638551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3b36-85ec-440f-8e5a-31fb2c87d6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.641019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:39.642094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.642544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.642962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.644397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:39.647445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:39.649654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 212/407] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:39 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:39.655361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.660261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5b7e3b36-85ec-440f-8e5a-31fb2c87d6ea None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:39.661701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5b7e3b36-85ec-440f-8e5a-31fb2c87d6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:39.698954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:39.737707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:39.762732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 196/408] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:39.764038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ab9a1-0919-44be-a01d-4586e6097406 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.764329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ab9a1-0919-44be-a01d-4586e6097406 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.764631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ab9a1-0919-44be-a01d-4586e6097406 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.765920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 197/409] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:39.771390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcf32a-7136-49ed-a93b-a1cadaacc62f None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.771681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcf32a-7136-49ed-a93b-a1cadaacc62f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.771959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcf32a-7136-49ed-a93b-a1cadaacc62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.790750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-dfbcf32a-7136-49ed-a93b-a1cadaacc62f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:39.793072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-dfbcf32a-7136-49ed-a93b-a1cadaacc62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:39.819439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea9554-755c-4455-8abd-627f5de7b80c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:39.863164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:39.864687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:39.865194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:39.865706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:39.867366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:39.868887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:39.872297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 213/410] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:39 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:39.874014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.879422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 198/411] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:39.880367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcf32a-7136-49ed-a93b-a1cadaacc62f None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:39.982638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:39.994860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69989d-35d9-48de-93f7-f7518e2c1841 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.030137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcf32a-7136-49ed-a93b-a1cadaacc62f None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.045040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.046134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.046622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.046768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:40.047197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.060180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:40.068194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:40.069254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:40.070050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:40.070660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 214/412] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:40.071783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:40.075664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.076844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.077346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.077819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912/groups/c89dde724c2f41dbbaab789161d80064/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.080358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] RBAC: Authorizing `identity:create_grant(project_id=807a635f1d50461091b40698766b4912, group_id=c89dde724c2f41dbbaab789161d80064, role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:40.082060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:40.101707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 199/413] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Jun 24 09:05:39 2022] PUT /identity/v3/projects/807a635f1d50461091b40698766b4912/groups/c89dde724c2f41dbbaab789161d80064/roles/cc730cc298404131a99c3547377a5e49 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:05:40.109788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:40.267894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab78bb-60fc-4290-848b-599340ee49b0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.312236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7a3f-ff09-4fd0-96cb-0d1d4f2f5857 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.324061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.325190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.325521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.325865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.340930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:40.352156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:40.353078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:40.354440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:40.358278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 215/414] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:40.361564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.361724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.361910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.362144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.362980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:40.363286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:40.364432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:40.364899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:40.376181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 200/415] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:40.380235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:40.383349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:40.432946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:40.481474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec4c48-c56f-40b4-95b2-460640aff38d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.517789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.518909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.519394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.519867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.520882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:40.521400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:40.522958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:40.533049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 216/416] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:40.539065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:40.558580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313fdd7-95a5-4b60-8f17-518b67b5fc54 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.632084 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:40.657296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.658184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.658471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.658751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] PATH_INFO: `/v3/groups/a4acf24e78e6440ca493247d9880f598` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.659956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] RBAC: Authorizing `identity:get_group(group_id=a4acf24e78e6440ca493247d9880f598)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:40.661425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:40.663362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 201/417] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:40 2022] GET /identity/v3/groups/a4acf24e78e6440ca493247d9880f598 => generated 229 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:40.668349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:40.669548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cf827-b023-45fa-8ba1-41e3c97cc2d8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.707091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.708109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.708616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.709200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.789052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:40.791696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:40.893112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:40.906312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 217/418] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:40.917519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:40.959740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e61e3e-ff90-4cfb-82a8-c94aee112220 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:40.961501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:40.995612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:40.996478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:40.996792 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:40.997071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] PATH_INFO: `/v3/projects/ccae62deda0a4d329baa9cb863e45e2d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:40.998362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] RBAC: Authorizing `identity:get_project(project_id=ccae62deda0a4d329baa9cb863e45e2d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:41.000228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:41.002002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 202/419] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:40 2022] GET /identity/v3/projects/ccae62deda0a4d329baa9cb863e45e2d => generated 344 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:41.013227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:41.121753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9d8c3-153c-49f3-8ae8-cc6904a3f55d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:41.175806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe881c4-4743-4b76-ab69-2ac932d2ee7d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:41.214429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:41.215403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.215694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.215972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.217282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:41.218815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:41.219652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75b4dc-0849-4bfe-8aa8-d64b64b98e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:41.220844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 218/420] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:40 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:41.221051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b4dc-0849-4bfe-8aa8-d64b64b98e5a None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.221587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b4dc-0849-4bfe-8aa8-d64b64b98e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.222076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b4dc-0849-4bfe-8aa8-d64b64b98e5a None admin] PATH_INFO: `/v3/projects/ccae62deda0a4d329baa9cb863e45e2d/groups/a4acf24e78e6440ca493247d9880f598/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.228905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:41.230316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b4dc-0849-4bfe-8aa8-d64b64b98e5a None admin] RBAC: Authorizing `identity:create_grant(project_id=ccae62deda0a4d329baa9cb863e45e2d, group_id=a4acf24e78e6440ca493247d9880f598, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:41.231968 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b4dc-0849-4bfe-8aa8-d64b64b98e5a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:41.258281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 203/421] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Jun 24 09:05:41 2022] PUT /identity/v3/projects/ccae62deda0a4d329baa9cb863e45e2d/groups/a4acf24e78e6440ca493247d9880f598/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:05:41.289944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:41.309940 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff430eeb-1f03-459d-a32f-8bdaa2aacf63 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.310251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff430eeb-1f03-459d-a32f-8bdaa2aacf63 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.310501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff430eeb-1f03-459d-a32f-8bdaa2aacf63 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.312330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 204/422] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:41.318848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757a232-14aa-41b4-a925-ca52ee0a5fc4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.319461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757a232-14aa-41b4-a925-ca52ee0a5fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.319941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757a232-14aa-41b4-a925-ca52ee0a5fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.350594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f757a232-14aa-41b4-a925-ca52ee0a5fc4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:41.354125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f757a232-14aa-41b4-a925-ca52ee0a5fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:41.402325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:41.405220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0f78c-1367-4386-81f9-edf547c08581 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:41.467983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 205/423] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:41.469834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:41.470795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.471158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.471499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.476007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:41.476007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:41.476007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 219/424] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:41 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:41.482073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757a232-14aa-41b4-a925-ca52ee0a5fc4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:41.495761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:41.602657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757a232-14aa-41b4-a925-ca52ee0a5fc4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:41.627435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c78009-0512-4293-ad60-6516c2428709 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:41.639179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:41.639837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.640118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.640389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.641250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:41.641556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:41.643220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:41.645541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:41.651559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 206/425] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:41 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:41.692393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:41.693168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.693452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.693733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.710485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:41.718631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:41.719522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:41.720802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 220/426] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:41.725647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:41.762728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:41.850424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4b388-dc13-41d1-a24f-230b8edfd2d0 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:41.888877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76164a-f82c-47f5-ada9-c0991e466e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:41.889560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76164a-f82c-47f5-ada9-c0991e466e11 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:41.889841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76164a-f82c-47f5-ada9-c0991e466e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:41.890116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76164a-f82c-47f5-ada9-c0991e466e11 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:41.890870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76164a-f82c-47f5-ada9-c0991e466e11 None admin] RBAC: Adding query filter params (name=member) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:41.891180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76164a-f82c-47f5-ada9-c0991e466e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:41.892588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76164a-f82c-47f5-ada9-c0991e466e11 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:41.902314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 207/427] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:41.907816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.034596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737a98e-60d0-4435-9938-59290d0e9b49 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:42.054362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902259f-6c8c-4c82-a571-256f8ec550b4 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.054683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902259f-6c8c-4c82-a571-256f8ec550b4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.060357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902259f-6c8c-4c82-a571-256f8ec550b4 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.060357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 208/428] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:42.062053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b43709-d189-4563-b8bf-b016be44d1f9 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.062566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b43709-d189-4563-b8bf-b016be44d1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.063033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b43709-d189-4563-b8bf-b016be44d1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.078711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:42.079229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.079516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.079797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.085347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-57b43709-d189-4563-b8bf-b016be44d1f9 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:42.086782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-57b43709-d189-4563-b8bf-b016be44d1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:42.122218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:42.161543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:42.162821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:42.166893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 209/429] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:42.196189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b43709-d189-4563-b8bf-b016be44d1f9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.216732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:42.277360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 221/430] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:42.278822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.316905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b43709-d189-4563-b8bf-b016be44d1f9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:42.350048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:42.358583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:42.359681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.360259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.360778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.372660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:42.382839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:42.383846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:42.385336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:42.386934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755f110-3b53-4f67-a44a-719bb57d28e8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:42.392714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 210/431] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:42.394625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.424878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:42.425748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.426027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.426303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.427605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:42.430421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:42.433409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 222/432] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:42 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:42.445251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.493623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:42.522602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423ce08-407c-4cef-ab32-7b3842fef119 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:42.565885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:42.566561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.566845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.567178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.567614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001a90c-12cd-4328-a9f4-ccd96eca98a1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:42.567874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:42.568182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:42.569584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:42.579029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 211/433] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:42.584441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.612849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d022cdf-1ec8-406f-9304-e372c0b9da77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:42.614319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022cdf-1ec8-406f-9304-e372c0b9da77 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.614744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022cdf-1ec8-406f-9304-e372c0b9da77 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.615019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022cdf-1ec8-406f-9304-e372c0b9da77 None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.616212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d022cdf-1ec8-406f-9304-e372c0b9da77 None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:42.617714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d022cdf-1ec8-406f-9304-e372c0b9da77 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:42.620048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 223/434] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:42 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:42.622622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e696c06-1f10-46d4-830d-3174e6686200 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.622807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e696c06-1f10-46d4-830d-3174e6686200 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.623038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e696c06-1f10-46d4-830d-3174e6686200 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.625288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 224/435] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:42.628267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e696c06-1f10-46d4-830d-3174e6686200 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.633235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:42.722817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7370021-4292-4e0f-b102-cca1d2312f58 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:42.761783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e696c06-1f10-46d4-830d-3174e6686200 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:42.762114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:42.762767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.763049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.763341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.774389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:42.798709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25077f2-f6a5-4742-a46a-087b600a133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:42.799365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25077f2-f6a5-4742-a46a-087b600a133b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.799651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25077f2-f6a5-4742-a46a-087b600a133b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.799928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25077f2-f6a5-4742-a46a-087b600a133b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.800863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25077f2-f6a5-4742-a46a-087b600a133b None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=0fc9d0c176a542f78847c87ede7b7b17, scope.project.id=807a635f1d50461091b40698766b4912) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:42.801180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25077f2-f6a5-4742-a46a-087b600a133b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:42.802670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25077f2-f6a5-4742-a46a-087b600a133b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:42.809731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 225/436] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:42 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=807a635f1d50461091b40698766b4912&role.id=0fc9d0c176a542f78847c87ede7b7b17 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:42.810688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e837ac-a134-4e5f-8095-9d2316f60f19 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:42.810975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e837ac-a134-4e5f-8095-9d2316f60f19 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:42.811250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e837ac-a134-4e5f-8095-9d2316f60f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:42.829568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a6e837ac-a134-4e5f-8095-9d2316f60f19 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:42.830720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a6e837ac-a134-4e5f-8095-9d2316f60f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:42.864614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:42.867821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:42.917010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:42.933201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 226/437] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:42.960002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e837ac-a134-4e5f-8095-9d2316f60f19 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:42.998815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 212/438] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:05:43.004665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.054282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:43.091343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e837ac-a134-4e5f-8095-9d2316f60f19 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.122429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dc67a-86a2-4818-ad88-10a16ca28251 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.128579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.129393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.129674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.129994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.144266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:43.154734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:43.155643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:43.156462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 227/439] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:43.159310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.160174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.160454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.161040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] PATH_INFO: `/v3/projects/473c53bf5c8740e4aaabb9c9465bd165` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.161565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.162377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] RBAC: Authorizing `identity:get_project(project_id=473c53bf5c8740e4aaabb9c9465bd165)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:43.163818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:43.165835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 213/440] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/projects/473c53bf5c8740e4aaabb9c9465bd165 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:43.170766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.226919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:43.274788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6591-7525-4a94-ba50-fe4ca7dea038 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.275761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056e802-c91f-4d14-bc9e-1c32d0647294 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.283678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:43.310864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.311086 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.311533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.311724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.311841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.312010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.312117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.312289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.313023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:43.313330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:43.313517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:43.314714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:43.314930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:43.316814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 214/441] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:43.322767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.324592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 228/442] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:43.329233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.427493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ed968-3a47-42e0-bb3f-a59879fbd9e1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.438771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ce88-86a1-46e5-9146-d9ffa17935e8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.461145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:43.466862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db65eb3-9d0b-49fb-841b-04308d17432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.467577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db65eb3-9d0b-49fb-841b-04308d17432f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.467862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db65eb3-9d0b-49fb-841b-04308d17432f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.468141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db65eb3-9d0b-49fb-841b-04308d17432f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.469009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db65eb3-9d0b-49fb-841b-04308d17432f None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=473c53bf5c8740e4aaabb9c9465bd165) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:43.469320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db65eb3-9d0b-49fb-841b-04308d17432f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:43.470758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db65eb3-9d0b-49fb-841b-04308d17432f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:43.476863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.477009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 215/443] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=473c53bf5c8740e4aaabb9c9465bd165&role.id=cc730cc298404131a99c3547377a5e49 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:43.477787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.478074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.478880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.562119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:43.563718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:43.662336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bc8e6-71e7-4f6a-acfd-685b76150b84 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.662844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bc8e6-71e7-4f6a-acfd-685b76150b84 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.662947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bc8e6-71e7-4f6a-acfd-685b76150b84 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.664260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 216/444] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:43.671869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619a431-738b-469e-a3dc-3e40e7a248c2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.672256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619a431-738b-469e-a3dc-3e40e7a248c2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.672631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619a431-738b-469e-a3dc-3e40e7a248c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.675468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 229/445] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:05:43.681033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.692145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3619a431-738b-469e-a3dc-3e40e7a248c2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:43.693440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3619a431-738b-469e-a3dc-3e40e7a248c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:43.708108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:43.732688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:43.776366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 217/446] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:43.799785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da88fca-e297-45b2-95ee-558facfdeff5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.801312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619a431-738b-469e-a3dc-3e40e7a248c2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.841600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.842511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.842794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.843072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] PATH_INFO: `/v3/projects/807a635f1d50461091b40698766b4912` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.844290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] RBAC: Authorizing `identity:get_project(project_id=807a635f1d50461091b40698766b4912)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:43.845773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:43.847485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 230/447] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/projects/807a635f1d50461091b40698766b4912 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:43.852920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:43.915640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619a431-738b-469e-a3dc-3e40e7a248c2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.957851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:43.958446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:43.959321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:43.959321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:43.959591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:43.970923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:43.974238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38c36-ad53-43e7-b598-a387a138bf89 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:43.978768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:43.979660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:43.980479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 218/448] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:43.986098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:44.022946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:44.023204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:44.023695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:44.024105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:44.024855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] PATH_INFO: `/v3/groups/c89dde724c2f41dbbaab789161d80064` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:44.026940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] RBAC: Authorizing `identity:get_group(group_id=c89dde724c2f41dbbaab789161d80064)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:44.028698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:44.032610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 231/449] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/groups/c89dde724c2f41dbbaab789161d80064 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:44.038239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:44.097401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431158-453f-4e26-9b6b-ab961d2b0550 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:44.130133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:44.130805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:44.131092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:44.131373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:44.132116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:44.132427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:44.133846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:44.137469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182e62b-f7bb-463d-acfc-0797daa1897e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:44.142938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 219/450] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:44.148056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:44.174114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:44.174778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:44.175071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:44.175352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:44.176213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] RBAC: Adding query filter params (group.id=c89dde724c2f41dbbaab789161d80064, role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=807a635f1d50461091b40698766b4912) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:44.176525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:44.177992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:44.184753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 232/451] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:44 2022] GET /identity/v3/role_assignments?group.id=c89dde724c2f41dbbaab789161d80064&scope.project.id=807a635f1d50461091b40698766b4912&role.id=cc730cc298404131a99c3547377a5e49 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:44.243144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:44.254204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3e5cf-5c3f-490d-bdd9-af3deb0f947d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:44.289098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:44.289886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:44.290173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:44.290452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:44.362048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:44.363499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:44.454904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:44.466092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 220/452] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:05:44.471860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:44.495549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:44.574286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0585d01-3a2c-4dcb-bf12-23e7febee22f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:44.607763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:44.608631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:44.608916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:44.609188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] PATH_INFO: `/v3/projects/ccae62deda0a4d329baa9cb863e45e2d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:44.610353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] RBAC: Authorizing `identity:get_project(project_id=ccae62deda0a4d329baa9cb863e45e2d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:44.611775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:44.613466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 233/453] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:05:44 2022] GET /identity/v3/projects/ccae62deda0a4d329baa9cb863e45e2d => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:44.620898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:44.716026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f21244-91c9-47c4-bb55-6c807721a5b3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:44.748619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54358c6c-24d4-48fa-9b8f-e8731b630556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:44.749409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358c6c-24d4-48fa-9b8f-e8731b630556 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:44.749690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358c6c-24d4-48fa-9b8f-e8731b630556 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:44.749971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358c6c-24d4-48fa-9b8f-e8731b630556 None admin] PATH_INFO: `/v3/groups/a4acf24e78e6440ca493247d9880f598` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:44.751127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54358c6c-24d4-48fa-9b8f-e8731b630556 None admin] RBAC: Authorizing `identity:get_group(group_id=a4acf24e78e6440ca493247d9880f598)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:44.752575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54358c6c-24d4-48fa-9b8f-e8731b630556 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:44.754164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 221/454] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Jun 24 09:05:44 2022] GET /identity/v3/groups/a4acf24e78e6440ca493247d9880f598 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:44.759785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:44.846059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:44.858981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519934b-745e-4556-9623-1bab101d2f53 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:44.891137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1cc3e-900a-458e-a870-43d0a7a85cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:44.891778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1cc3e-900a-458e-a870-43d0a7a85cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:44.892063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1cc3e-900a-458e-a870-43d0a7a85cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:44.892336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1cc3e-900a-458e-a870-43d0a7a85cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:44.893180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1cc3e-900a-458e-a870-43d0a7a85cf2 None admin] RBAC: Adding query filter params (group.id=a4acf24e78e6440ca493247d9880f598, role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=ccae62deda0a4d329baa9cb863e45e2d) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:44.893485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1cc3e-900a-458e-a870-43d0a7a85cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:44.894895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1cc3e-900a-458e-a870-43d0a7a85cf2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:44.901088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 234/455] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Jun 24 09:05:44 2022] GET /identity/v3/role_assignments?group.id=a4acf24e78e6440ca493247d9880f598&scope.project.id=ccae62deda0a4d329baa9cb863e45e2d&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:47.278363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd8dff-b1d0-4a80-9b4f-9b453858b5ec None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.279436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd8dff-b1d0-4a80-9b4f-9b453858b5ec None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.279920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd8dff-b1d0-4a80-9b4f-9b453858b5ec None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.281626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 222/456] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:47.287194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5f479-0bf3-4bfa-930c-79382215b42e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.288006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5f479-0bf3-4bfa-930c-79382215b42e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.288513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5f479-0bf3-4bfa-930c-79382215b42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.313391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-01a5f479-0bf3-4bfa-930c-79382215b42e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:47.314959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-01a5f479-0bf3-4bfa-930c-79382215b42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:47.392968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 235/457] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:47.429752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd8dff-b1d0-4a80-9b4f-9b453858b5ec None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:47.505743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:47.555941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd8dff-b1d0-4a80-9b4f-9b453858b5ec None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:47.567815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8174e27-23d3-40a0-9bc3-f36194f2c9c8 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.568383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8174e27-23d3-40a0-9bc3-f36194f2c9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.568890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8174e27-23d3-40a0-9bc3-f36194f2c9c8 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.570589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 236/458] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:47.575743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81c586-d3b4-4e93-bb26-b67ba37cf148 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.576247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81c586-d3b4-4e93-bb26-b67ba37cf148 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.576761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81c586-d3b4-4e93-bb26-b67ba37cf148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.596677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2c81c586-d3b4-4e93-bb26-b67ba37cf148 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:47.598256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2c81c586-d3b4-4e93-bb26-b67ba37cf148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:47.603490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373962ed-4118-4d94-b79b-16c02025feb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:47.604292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373962ed-4118-4d94-b79b-16c02025feb4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.604589 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373962ed-4118-4d94-b79b-16c02025feb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.605011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373962ed-4118-4d94-b79b-16c02025feb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.618516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373962ed-4118-4d94-b79b-16c02025feb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:47.620518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373962ed-4118-4d94-b79b-16c02025feb4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-373962ed-4118-4d94-b79b-16c02025feb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:47.644007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:47.644921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:47.645758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:47.645758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:47.645758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:47.645758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:47.645758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 223/459] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:47.648886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ac15c-21a4-4d84-bdb9-697cf4bf3957 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.649373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ac15c-21a4-4d84-bdb9-697cf4bf3957 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.649831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ac15c-21a4-4d84-bdb9-697cf4bf3957 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.651357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 224/460] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:47.653548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ac15c-21a4-4d84-bdb9-697cf4bf3957 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:47.678629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 237/461] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:47.680297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d57686-5ccd-4545-82d2-bb6ddd030642 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.680827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d57686-5ccd-4545-82d2-bb6ddd030642 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.681840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d57686-5ccd-4545-82d2-bb6ddd030642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.698824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-87d57686-5ccd-4545-82d2-bb6ddd030642 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:47.700297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-87d57686-5ccd-4545-82d2-bb6ddd030642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:47.712588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:47.779130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 238/462] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:47.780371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d57686-5ccd-4545-82d2-bb6ddd030642 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:47.799255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ac15c-21a4-4d84-bdb9-697cf4bf3957 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:47.837027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:47.845373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:47.846485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.846984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.847453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.848858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:47.850803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:47.851429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:47.864881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 225/463] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:47.866489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:47.920453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d57686-5ccd-4545-82d2-bb6ddd030642 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:47.959912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:47.961185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:47.961739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:47.962206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:47.974730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:47.977673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:47.980001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:47.997127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e27f9f-eab5-43e1-9d0b-5f9a88a93214 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:48.003269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:48.004117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:48.005397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:48.005397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:48.005397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:48.005397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:48.005397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 239/464] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:48.005397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:48.046802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:48.048068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.048835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.049328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.065140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:48.067119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:48.086279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:48.087163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:48.088069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:48.088069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:48.088069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:48.088069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:48.090381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 226/465] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:48.091444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:48.127593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:48.130511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1fc79-509a-4716-ac80-a22cf2801c1c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:48.168125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:48.169218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.169782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.170286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.171240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:48.172934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:48.200016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 240/466] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:48 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:48.201255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:48.245209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd8d22-8eaf-45b4-a7ca-7eb027cdcf6a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:48.329285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:48.331010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.332822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.333426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:48.336846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.339014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:48.340179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:48.343722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:48.376325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 227/467] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:48.381983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:48.425997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ab2db-ba6b-4cef-8b60-11a52f96050b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:48.469738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75254412-f50a-471f-915b-944c9ecf1b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:48.470551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75254412-f50a-471f-915b-944c9ecf1b4d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.470936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75254412-f50a-471f-915b-944c9ecf1b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.471425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75254412-f50a-471f-915b-944c9ecf1b4d None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.473029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75254412-f50a-471f-915b-944c9ecf1b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:48.473029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75254412-f50a-471f-915b-944c9ecf1b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:48.474229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75254412-f50a-471f-915b-944c9ecf1b4d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:48.486166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:48.488670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 241/468] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:48.490170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe2a46-7ae1-4bcc-8451-9c0960200060 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.492736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe2a46-7ae1-4bcc-8451-9c0960200060 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.492736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe2a46-7ae1-4bcc-8451-9c0960200060 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.492736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 242/469] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:48.497419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe2a46-7ae1-4bcc-8451-9c0960200060 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:48.508192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8769d57-4e03-4890-9097-3550f223379a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:48.556334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d75805-c805-498e-a076-9160e7797adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:48.557041 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d75805-c805-498e-a076-9160e7797adc None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.557327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d75805-c805-498e-a076-9160e7797adc None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.557871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d75805-c805-498e-a076-9160e7797adc None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.558584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d75805-c805-498e-a076-9160e7797adc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:48.560035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d75805-c805-498e-a076-9160e7797adc None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:48.582699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 228/470] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:48 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:48.588881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54e82c-4f10-44df-853a-d13a27e58d92 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.588881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54e82c-4f10-44df-853a-d13a27e58d92 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.588881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54e82c-4f10-44df-853a-d13a27e58d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.623402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9d54e82c-4f10-44df-853a-d13a27e58d92 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:48.624567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9d54e82c-4f10-44df-853a-d13a27e58d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:48.703178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe2a46-7ae1-4bcc-8451-9c0960200060 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:48.730579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:48.780739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 229/471] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:48.782042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:48.783004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.783420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.783746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.784709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:48.786663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:48.808719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 243/472] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:48 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:48.816332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669f026-be33-40aa-a130-63c6028a2b65 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.816805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669f026-be33-40aa-a130-63c6028a2b65 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.817097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669f026-be33-40aa-a130-63c6028a2b65 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.818944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 230/473] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:48.822742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:48.824998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e470abe-716e-4fb7-806c-02b700f4b4a6 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:48.825294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e470abe-716e-4fb7-806c-02b700f4b4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:48.825580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e470abe-716e-4fb7-806c-02b700f4b4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:48.854755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-0e470abe-716e-4fb7-806c-02b700f4b4a6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:48.856588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0e470abe-716e-4fb7-806c-02b700f4b4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:48.964036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:48.977149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 231/474] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:48.985799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ea6ce-6caa-4077-958a-38ebde090b01 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:48.999020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e470abe-716e-4fb7-806c-02b700f4b4a6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:49.035188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:49.035994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:49.036751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:49.036751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:49.049291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:49.050773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:49.069969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:49.070877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:49.071708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:49.071708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:49.071708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:49.071708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:49.071708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 244/475] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:49.078095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:49.108259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:49.128747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e470abe-716e-4fb7-806c-02b700f4b4a6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:49.168138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:49.168862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:49.169147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:49.169425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:49.170301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:49.172259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:49.179431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.sql.core [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 09:05:49.179431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 09:05:49.179431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [parameters: {'id': '3c9cb68aa53746218f95d10bbfe98327', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 09:05:49.179431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80156) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result.read() Jun 24 09:05:49.201458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:05:49.202838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application yield resource Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise self.value Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application session.commit() Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 09:05:49.204690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.session.flush() Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application compat.raise_( Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 09:05:49.205567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:05:49.206436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result.read() Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:05:49.207251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [parameters: {'id': '3c9cb68aa53746218f95d10bbfe98327', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:49.208039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 09:05:49.212003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:49.216550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 232/476] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Jun 24 09:05:48 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:49.216550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:49.221467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc4f6-62f4-49af-8776-75caeec66c4d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:49.266648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:49.267874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:49.268683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:49.269170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:49.270159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:49.270666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:49.273049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:49.286704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:49.290667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 245/477] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:49.300362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:49.347816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de3ec-8f26-4fe1-85b1-53f92514254c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:49.409880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:49.410713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:49.410993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:49.411270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:49.438209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:49.443942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:49.461566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:49.462546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:05:49.463345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:49.475889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 233/478] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:05:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:49.481866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:49.484175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599910e-590a-4f15-a10e-d93a7d31d823 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:49.569147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261070bb-d6ee-4f13-be66-523ab086ce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:49.569923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261070bb-d6ee-4f13-be66-523ab086ce3f None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:49.570236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261070bb-d6ee-4f13-be66-523ab086ce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:49.572299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261070bb-d6ee-4f13-be66-523ab086ce3f None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:49.573127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261070bb-d6ee-4f13-be66-523ab086ce3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:49.576967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261070bb-d6ee-4f13-be66-523ab086ce3f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:49.626786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 246/479] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:49 2022] POST /identity/v3/users => generated 249 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:05:49.644198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1b495-5fcb-47a4-a5f6-2df24130a4dd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:49.670449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:49.688196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df100d9f-6aa0-4d07-b321-f73c170152d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:49.689204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df100d9f-6aa0-4d07-b321-f73c170152d7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:49.689696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df100d9f-6aa0-4d07-b321-f73c170152d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:49.690156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df100d9f-6aa0-4d07-b321-f73c170152d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:49.691105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df100d9f-6aa0-4d07-b321-f73c170152d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:49.691587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df100d9f-6aa0-4d07-b321-f73c170152d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:49.693161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df100d9f-6aa0-4d07-b321-f73c170152d7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:49.711146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 234/480] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:50.496838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb4601-9eff-46c9-858b-639eccaa29e7 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:50.497714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb4601-9eff-46c9-858b-639eccaa29e7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:50.498195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb4601-9eff-46c9-858b-639eccaa29e7 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:50.499760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 247/481] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:50.514055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb38293-b29a-41dd-b3f2-5043cb2e7c66 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:50.514858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb38293-b29a-41dd-b3f2-5043cb2e7c66 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:50.515279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb38293-b29a-41dd-b3f2-5043cb2e7c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:50.537452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8bb38293-b29a-41dd-b3f2-5043cb2e7c66 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:50.538845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8bb38293-b29a-41dd-b3f2-5043cb2e7c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:50.622821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 235/482] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:50.658586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb4601-9eff-46c9-858b-639eccaa29e7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:50.796523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb4601-9eff-46c9-858b-639eccaa29e7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:50.838608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:50.839761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:50.840226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:50.840672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:50.842702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:50.855359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:50.863363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:50.864432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:50.865517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:50.867734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 248/483] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:50.873132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb38293-b29a-41dd-b3f2-5043cb2e7c66 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:51.016241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb38293-b29a-41dd-b3f2-5043cb2e7c66 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:51.042677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:51.072912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39040db3-cbde-45a6-95d8-e61ec2eed342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:51.073516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39040db3-cbde-45a6-95d8-e61ec2eed342 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.073804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39040db3-cbde-45a6-95d8-e61ec2eed342 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.074085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39040db3-cbde-45a6-95d8-e61ec2eed342 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.074840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39040db3-cbde-45a6-95d8-e61ec2eed342 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:51.075151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39040db3-cbde-45a6-95d8-e61ec2eed342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:51.076521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39040db3-cbde-45a6-95d8-e61ec2eed342 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:51.092984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 236/484] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:51.098508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:51.249623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71118a69-faab-4aaf-b220-477fb9cff644 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:51.256623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87342-0039-4f69-8247-bb1c3271d634 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.256623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87342-0039-4f69-8247-bb1c3271d634 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.256623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87342-0039-4f69-8247-bb1c3271d634 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.256623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 237/485] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:05:51.264586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c9d2-8e7a-4959-b239-028f012d9fd6 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.264586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c9d2-8e7a-4959-b239-028f012d9fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.264586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c9d2-8e7a-4959-b239-028f012d9fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.291661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a684c9d2-8e7a-4959-b239-028f012d9fd6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:51.292884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a684c9d2-8e7a-4959-b239-028f012d9fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:51.316772 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:51.317566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.317846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.318135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.337527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:51.340323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:51.359782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:51.360726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:51.361572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:51.361572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:51.361572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:51.361572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:51.362206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 249/486] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:51.370989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:51.429791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:51.456412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 238/487] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:05:51.490735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b38955-fde8-443d-b9b0-558a1317fd9d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:51.495691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c9d2-8e7a-4959-b239-028f012d9fd6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:51.528921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:51.530319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.530968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.531384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.532241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:51.532737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:51.534437 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:51.549730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 250/488] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:51.557981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:51.666468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c9d2-8e7a-4959-b239-028f012d9fd6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:51.675147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:51.708420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33950b9-d5c1-405d-9ae3-f161826d18a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:51.711860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33950b9-d5c1-405d-9ae3-f161826d18a3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.712146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33950b9-d5c1-405d-9ae3-f161826d18a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.712967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33950b9-d5c1-405d-9ae3-f161826d18a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.736812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33950b9-d5c1-405d-9ae3-f161826d18a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-f33950b9-d5c1-405d-9ae3-f161826d18a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:51.747479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:51.748399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:51.749242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:51.753556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 239/489] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:51.755631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebac20-29d9-4008-becf-fad7a77294f4 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.756201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebac20-29d9-4008-becf-fad7a77294f4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.756487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebac20-29d9-4008-becf-fad7a77294f4 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.758603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 240/490] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:51.761286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebac20-29d9-4008-becf-fad7a77294f4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:51.774845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb763739-de3b-48f7-b774-27c9eaa89025 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:51.784135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:51.816301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa444e-0d88-4994-80a2-f079d0f4335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:51.817219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa444e-0d88-4994-80a2-f079d0f4335d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.817655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa444e-0d88-4994-80a2-f079d0f4335d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.817988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa444e-0d88-4994-80a2-f079d0f4335d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.826477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa444e-0d88-4994-80a2-f079d0f4335d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:51.828320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa444e-0d88-4994-80a2-f079d0f4335d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ecfa444e-0d88-4994-80a2-f079d0f4335d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:05:51.848465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:51.849432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:51.850332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 251/491] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:05:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:51.851670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c3dfa-1f1e-495f-aa79-56c774a53c67 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.852004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c3dfa-1f1e-495f-aa79-56c774a53c67 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.852286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c3dfa-1f1e-495f-aa79-56c774a53c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.876196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ad2c3dfa-1f1e-495f-aa79-56c774a53c67 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:51.877271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebac20-29d9-4008-becf-fad7a77294f4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:51.877541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ad2c3dfa-1f1e-495f-aa79-56c774a53c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:51.915105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:51.915825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:51.916111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:51.916529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:51.917365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:51.917770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:51.919257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:51.928118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 241/492] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:51.929072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:51.964707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 252/493] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:51.965716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:51.966385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c3dfa-1f1e-495f-aa79-56c774a53c67 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.035997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07250d15-bd99-44cf-ae59-53af22cd9a10 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.071079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.072409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.072793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.073147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.074160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:52.074536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.076082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.090002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.096968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c3dfa-1f1e-495f-aa79-56c774a53c67 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.115134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 242/494] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:05:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:52.120949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.138539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.139353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.139635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.139915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.151544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.153269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:52.172473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:52.173368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:52.174231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:52.174231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:52.174231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:52.174231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:52.174231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 253/495] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:52.178028 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.212277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.229007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbf0d8-57c2-4061-91df-caf5daabaf64 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.271095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.271906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.272191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.272468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.282989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:52.290770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:52.291629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:52.292434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 243/496] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:52.293694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.299268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7ab41-28d1-4928-a3df-e6b19768e3e6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.336500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3805b3-96c2-4ef9-a1d2-b89567aa011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.337179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3805b3-96c2-4ef9-a1d2-b89567aa011b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.337492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3805b3-96c2-4ef9-a1d2-b89567aa011b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.337774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3805b3-96c2-4ef9-a1d2-b89567aa011b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.356644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.396587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cad877-0479-488e-90e5-fed800ac02e2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.407189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3805b3-96c2-4ef9-a1d2-b89567aa011b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.408670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3805b3-96c2-4ef9-a1d2-b89567aa011b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.429640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef986b99-b840-4878-8283-42a9b3257157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.430214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986b99-b840-4878-8283-42a9b3257157 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.430495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986b99-b840-4878-8283-42a9b3257157 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.430780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986b99-b840-4878-8283-42a9b3257157 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.432034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef986b99-b840-4878-8283-42a9b3257157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:52.432342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef986b99-b840-4878-8283-42a9b3257157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.433989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef986b99-b840-4878-8283-42a9b3257157 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.445381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 244/497] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:52.446660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee049ac-f3a8-4939-a25d-98948cf69c84 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.447513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee049ac-f3a8-4939-a25d-98948cf69c84 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.447789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee049ac-f3a8-4939-a25d-98948cf69c84 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.449039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 245/498] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:52.449723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee049ac-f3a8-4939-a25d-98948cf69c84 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.459318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.513648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 254/499] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:52.514970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40d637-cfe8-493c-9b46-35bf997f7133 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.515254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40d637-cfe8-493c-9b46-35bf997f7133 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.515762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40d637-cfe8-493c-9b46-35bf997f7133 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.517095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 255/500] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:52.517729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40d637-cfe8-493c-9b46-35bf997f7133 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.562103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee049ac-f3a8-4939-a25d-98948cf69c84 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.571529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.595392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deafee7-8dd5-41ac-b164-488bb87cb3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.596061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deafee7-8dd5-41ac-b164-488bb87cb3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.596336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deafee7-8dd5-41ac-b164-488bb87cb3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.596642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deafee7-8dd5-41ac-b164-488bb87cb3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.597387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deafee7-8dd5-41ac-b164-488bb87cb3d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:52.597693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deafee7-8dd5-41ac-b164-488bb87cb3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.599052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deafee7-8dd5-41ac-b164-488bb87cb3d6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.607534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 246/501] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:52.608716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535411d-fa8c-4ef8-8ae5-f1fca3a5baea None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.609007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535411d-fa8c-4ef8-8ae5-f1fca3a5baea None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.609341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535411d-fa8c-4ef8-8ae5-f1fca3a5baea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.626399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-6535411d-fa8c-4ef8-8ae5-f1fca3a5baea None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:52.628152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-6535411d-fa8c-4ef8-8ae5-f1fca3a5baea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:52.635606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40d637-cfe8-493c-9b46-35bf997f7133 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.671875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.672703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.672983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.673250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.680433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.682683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.691344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:05:52.697074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:52.697965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 256/502] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:52.699022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.708843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 247/503] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:52.710070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3115c5-90d1-4c1a-b1cb-4368b8e080e3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.710776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3115c5-90d1-4c1a-b1cb-4368b8e080e3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.711062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3115c5-90d1-4c1a-b1cb-4368b8e080e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.731844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-db3115c5-90d1-4c1a-b1cb-4368b8e080e3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:52.733078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-db3115c5-90d1-4c1a-b1cb-4368b8e080e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:52.799010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 248/504] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:52.801458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3115c5-90d1-4c1a-b1cb-4368b8e080e3 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.802880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e76a9-4248-4a7e-a456-7dd934a46d03 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.812938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.841129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.841926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.842210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.842483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.852512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.854630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:52.871820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:52.872858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:52.873710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:52.873710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:52.873710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:52.873710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:52.873710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 257/505] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:52.874180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:52.903963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3115c5-90d1-4c1a-b1cb-4368b8e080e3 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.932021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:52.940528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:52.941356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:52.941623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:52.941892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:52.951606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:52.953434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:52.975187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de2784-1211-4d5d-a6a3-33e3091ab4b9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:52.975747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:52.976839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:52.977829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:52.977829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:52.977829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:52.977829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:52.977829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 249/506] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:52.978777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.015169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.015842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.016148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.016444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.017340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:53.017639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.020505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.057724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 258/507] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:53.058560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.077528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.085971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f12ba6-45d0-494f-bad1-2fab577923da None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.121771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.122559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.122838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.123106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.133594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.135642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:53.152169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:53.153169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:53.154174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:53.154174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:53.154174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:53.154174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:53.154174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 250/508] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:53.154502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.161827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89464b-94ee-4bb7-8af0-865144b707fe None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.186672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.195483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.196280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.196585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.197494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.207442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:53.215045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:53.222208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 259/509] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:53.223792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.257048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed8040-9ea9-4b21-8492-dd8f28117572 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.291954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.292649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.292931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.293263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.294277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:53.294584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.296399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.301930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.311581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 251/510] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:53.312470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.332957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483eda9-538f-413b-a49c-80a0260d180e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.365754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.366412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.366702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.366976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.367681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:53.367983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.369978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.381832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 260/511] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:53.382619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.407297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.417063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b71a3-f3fb-44a9-b8eb-1d3babe96611 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.451396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.451968 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.452247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.452520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.484809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c1274-2a85-4bf0-ac40-cb2d5cb0ea90 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.519663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.521148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.527294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.527950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.528230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.528503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.529244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:53.529544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.530897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.537244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.543634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 261/512] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:53.544474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.613035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.623214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 252/513] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:53.624035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.646295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9497-14fd-4879-9d82-41124d290373 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.683208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.683866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.684147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.684427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.685197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:53.685500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.686872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.694736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 262/514] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:53.695619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.729494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc867-b32d-48b1-8084-051a5de3ef3d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.754066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.762812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.763601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.763886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.764164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.770739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.772197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:05:53.787291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:53.788162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:53.789372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 253/515] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:53.789833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.801329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef8e11-884a-4f6b-ad3a-1863634629bc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.835137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.835928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.836225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.836506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] PATH_INFO: `/v3/users/glance` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.845332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.845829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:53.846791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:05:53.862529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:53.863506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 263/516] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:53.864915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.896654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a62fc-2c5e-458c-a8e9-d99ff5f1ca61 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:53.932386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:53.933335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:53.933622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:53.933904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:53.934611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:53.936060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:53.955099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 254/517] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:05:53 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:05:53.955948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:53.986404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4288d-2d57-4321-9de2-497ec597ccfa None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.013674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.036175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6a39-6719-447e-85af-747254079d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.037039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6a39-6719-447e-85af-747254079d08 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.037322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6a39-6719-447e-85af-747254079d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.037594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6a39-6719-447e-85af-747254079d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.051655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af6a39-6719-447e-85af-747254079d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.053424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af6a39-6719-447e-85af-747254079d08 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-47af6a39-6719-447e-85af-747254079d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:54.072844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:54.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:54.074927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:54.074927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:54.074927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:54.074927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:54.081247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 264/518] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:54.082212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6a39-6719-447e-85af-747254079d08 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.089071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f2bf9-2c63-4395-92ff-e5033cc1915f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.099560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.131551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.132357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.132702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.132983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.146501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.148061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:54.171244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:54.172203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:54.173074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:54.173074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:54.173074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:54.173074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:54.173074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 255/519] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:54.174003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.211512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6a39-6719-447e-85af-747254079d08 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.248116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.248746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.249035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.249316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.250116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:54.250427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.251864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.271146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 265/520] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:54.272073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.290404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82981338-8a63-46f0-b3a8-de7e27a9d435 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.312770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.330486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.331471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.331723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.332109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.333121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:54.333442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.334872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.354615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 256/521] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:54.355459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.377883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.399211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d31a-63fc-4576-9a31-39d037492704 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.440536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.441342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.441695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.442508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.443385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:54.444171 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.445989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.459947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 266/522] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:54.460819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.475852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62042ebd-c91c-4716-ab59-cf2152992850 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.520536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.521407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.522137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.522716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.523636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:54.524201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.526042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.546103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 257/523] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:54.546103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.570267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.581676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e94a3c-f3f4-4486-b3b8-54aea0091e68 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.620242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.620954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.621236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.621513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.622339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=48647bb993344a8b91cec7f623aea007) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:54.622648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.624087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.629894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 267/524] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/role_assignments?user.id=48647bb993344a8b91cec7f623aea007&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:54.630682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.652985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.690109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd349c4-b9e6-4ffe-ac74-3f5e34d6d43d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.736827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.738405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.739008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.739578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.763830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d89f0-5a8b-4986-994b-e0b520fdd54b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:54.808067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:54.809210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:54.809497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:54.810218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:54.818363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.820066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.825930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:54.827436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:05:54.835678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:54.836607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:54.837625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:54.840297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 268/525] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:54.841401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.859974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.922804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:54.947907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 258/526] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:54.953214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:54.954610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bcda4-183b-467b-87cf-4611b7f80517 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.000388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.002114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.002672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.003121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.012860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.014525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:05:55.043966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:55.044892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:05:55.045783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:55.049215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 269/527] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:55.050149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:55.086078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ad9f-b07a-43d8-912b-e81e86a1d7ac None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.128269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.129254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.129796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.130506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.131418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:55.131806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.133587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.150577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:55.160691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 259/528] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:55.162071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:55.179089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb57c46-f4ca-4544-adac-4bac52281d16 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.208225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:55.227235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.228726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.229470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.230599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.246373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.248083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:55.272121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:55.273210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:55.274198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:55.274198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:55.274198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:55.274198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:55.276620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 270/529] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:55.277786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:55.282498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e06625-641b-4cfd-9b13-3e1112ac9fad None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.320114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.321336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.322402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.322695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.323543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:55.323915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.325926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.348291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 260/530] 10.222.0.52 () {56 vars in 1139 bytes} [Fri Jun 24 09:05:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:55.349235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:55.417952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95c458-388b-4b0e-b1bf-3983481c1758 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.454931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:55.457619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:55.462896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f562-227e-42bb-9538-5eb03a23d3b8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.468402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.469100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.469382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.469661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.470501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=31bf0e50e20b47d0a63662bbecce833d) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:55.470816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.472418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.484340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 271/531] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:55 2022] GET /identity/v3/role_assignments?user.id=31bf0e50e20b47d0a63662bbecce833d&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:55.489502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:55.531111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.532192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.532697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.533165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.534117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:55.534628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.536296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.561951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 261/532] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:55.568625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:55.699506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b8f0c-3fe1-4ee7-9460-60ed8b247460 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.750790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935e7b0-a62c-46cd-92ff-078bb81671a6 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:55.760665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b51db-567c-4abc-8828-2f07ddbd8309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.761337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b51db-567c-4abc-8828-2f07ddbd8309 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.762342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b51db-567c-4abc-8828-2f07ddbd8309 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.762342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b51db-567c-4abc-8828-2f07ddbd8309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.792032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:55.792925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:55.793208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:55.793536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:55.811053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.812856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:05:55.844549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:55.845461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:55.846295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:55.853019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 262/533] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:55.856510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:55.882108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:55.897766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b51db-567c-4abc-8828-2f07ddbd8309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:55.899239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b51db-567c-4abc-8828-2f07ddbd8309 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:55.904321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:56.004308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7846c7-42a2-4b4f-ad9e-9f60ce5ff161 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:56.040825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b30f-d7a5-4d88-8dd5-b762f8897d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:56.041495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b30f-d7a5-4d88-8dd5-b762f8897d6b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.041777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b30f-d7a5-4d88-8dd5-b762f8897d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.042054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b30f-d7a5-4d88-8dd5-b762f8897d6b None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.042844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906b30f-d7a5-4d88-8dd5-b762f8897d6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:56.043154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906b30f-d7a5-4d88-8dd5-b762f8897d6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:56.044616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906b30f-d7a5-4d88-8dd5-b762f8897d6b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:56.062578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 263/534] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:05:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:56.064010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efebda-94e3-4608-969e-a29f0950a326 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.064302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efebda-94e3-4608-969e-a29f0950a326 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.064623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efebda-94e3-4608-969e-a29f0950a326 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.066025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 264/535] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:56.067213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efebda-94e3-4608-969e-a29f0950a326 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:56.095817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 272/536] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:56.097191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327da42-4dae-4921-9264-e174c801b775 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.097484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327da42-4dae-4921-9264-e174c801b775 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.097765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327da42-4dae-4921-9264-e174c801b775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.114088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9327da42-4dae-4921-9264-e174c801b775 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:56.115290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9327da42-4dae-4921-9264-e174c801b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:56.150955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:56.176869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efebda-94e3-4608-969e-a29f0950a326 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:56.192812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 273/537] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:56.193726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327da42-4dae-4921-9264-e174c801b775 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:56.213275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:56.213936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.214221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.214498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.215326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=28a07f47b8954418a5a86f344fb7fbe6) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:56.215636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:56.217164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:56.228698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 265/538] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/role_assignments?user.id=28a07f47b8954418a5a86f344fb7fbe6&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:56.233144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:56.240265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:56.321260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327da42-4dae-4921-9264-e174c801b775 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:56.361472 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:56.362330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.362620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.362961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.375994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:56.377627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:56.397195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:56.398181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:56.399007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:56.399007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:56.399007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:56.399007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:56.399007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 274/539] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:56.404336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:56.498513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a13aef-f44b-45ed-9ee5-75845d88600e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:56.538227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb3d1-334b-40df-a089-42a7e7559dea None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:56.540887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:56.576022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:56.576639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.576927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.577212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.578069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:56.578379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:56.579884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:56.592761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 275/540] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:56.599364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:56.600187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.600475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.600802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.603744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:56.609108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:56.615286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:56.625437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:56.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:56.627546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:56.629298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 266/541] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:56.647119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:56.770616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751d98d-66ea-46a9-bc7a-5a4726029aa0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:56.822455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:56.823583 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.824884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.825312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.832965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:56.848904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:56.851024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:56.863920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47e94-7522-4059-84da-17e932ad20f7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:05:56.881950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:56.883477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:56.885112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:56.887481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 276/542] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:56.892954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:56.904412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:56.905236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:56.905558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:56.906438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:56.906656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:56.906964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:56.908359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:56.917781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 267/543] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:56.923823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.057693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:57.081975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c87578-8857-416b-ab27-d90dc880d625 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.104179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19d7a8-98e6-4b7a-bec5-4ba4624530f5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.121133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adbe3a-3fb7-4e17-9c0c-5f608b14638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.122151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adbe3a-3fb7-4e17-9c0c-5f608b14638d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.122603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adbe3a-3fb7-4e17-9c0c-5f608b14638d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.123040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adbe3a-3fb7-4e17-9c0c-5f608b14638d None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.123967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adbe3a-3fb7-4e17-9c0c-5f608b14638d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:57.124420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adbe3a-3fb7-4e17-9c0c-5f608b14638d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:57.126147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adbe3a-3fb7-4e17-9c0c-5f608b14638d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:57.138828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 277/544] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:57.138975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765b8d7-a8e3-43f7-ba5c-593e1c92087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.139709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b8d7-a8e3-43f7-ba5c-593e1c92087d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.139992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b8d7-a8e3-43f7-ba5c-593e1c92087d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.140269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765b8d7-a8e3-43f7-ba5c-593e1c92087d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.141113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d85a0b-fce5-4223-b9c2-f12a8df532bb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.141620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d85a0b-fce5-4223-b9c2-f12a8df532bb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.142220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d85a0b-fce5-4223-b9c2-f12a8df532bb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.144060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 278/545] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:57.145007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d85a0b-fce5-4223-b9c2-f12a8df532bb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.159135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765b8d7-a8e3-43f7-ba5c-593e1c92087d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:57.160625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765b8d7-a8e3-43f7-ba5c-593e1c92087d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-7765b8d7-a8e3-43f7-ba5c-593e1c92087d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:57.186051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:57.186915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:57.187757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:57.187757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:57.187757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:57.187757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:57.187757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 268/546] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:57.188503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e291c-bc5e-4413-87fa-41978c067db0 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.188712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e291c-bc5e-4413-87fa-41978c067db0 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.188988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e291c-bc5e-4413-87fa-41978c067db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.211946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8f2e291c-bc5e-4413-87fa-41978c067db0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:57.213234 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8f2e291c-bc5e-4413-87fa-41978c067db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:57.216547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:57.273120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d85a0b-fce5-4223-b9c2-f12a8df532bb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.307078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 269/547] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:57.308280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e291c-bc5e-4413-87fa-41978c067db0 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.313184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.314163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.314672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.315204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.316270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=ff6a58944beb49519f991a753adaa521) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:57.316840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:57.318547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:57.325280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 279/548] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:57 2022] GET /identity/v3/role_assignments?user.id=ff6a58944beb49519f991a753adaa521&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:57.336917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.362086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:57.430988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e291c-bc5e-4413-87fa-41978c067db0 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.478798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.479621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.479911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.480296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.481081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:57.481395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:57.482902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:57.500908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 270/549] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:57.502369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.565098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6c12d-89ac-46d4-85e8-f5cb082490d4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.597284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:57.632940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.633661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.633954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.634258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.644210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a10368-ce65-4feb-9523-9511a4144ec7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.648880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:57.660159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:57.670888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:57.672286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:57.673319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 280/550] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:05:57.681587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.715961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.716806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.717139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.717471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.729235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:57.730681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:05:57.762806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:57.765103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:57.767520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 271/551] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:05:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:57.774011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.812465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d22b-45e9-435f-93ac-2ddb645b790d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.849291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.849971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.850258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.850543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.851296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:57.851603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:57.853047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:57.862239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 281/552] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:57.871100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:57.923068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78772bb7-80c1-480d-87c6-c79fd85a95a3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:57.966211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:57.966896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:57.967185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:57.967457 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:57.968300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:57.968629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:57.970068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:57.975051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.009690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 272/553] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Jun 24 09:05:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:05:58.015401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.059252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.082471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86806c92-3fab-4446-a817-f410ecef9354 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.120324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf67d8b-ab39-4db3-b32d-356696cbc282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.121263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf67d8b-ab39-4db3-b32d-356696cbc282 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.121549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf67d8b-ab39-4db3-b32d-356696cbc282 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.121827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf67d8b-ab39-4db3-b32d-356696cbc282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.134071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf67d8b-ab39-4db3-b32d-356696cbc282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:58.135770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf67d8b-ab39-4db3-b32d-356696cbc282 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:58.141171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ad00-6a92-4743-9ef2-bd6c509b52d4 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-aaf67d8b-ab39-4db3-b32d-356696cbc282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:58.155344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:58.156282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:58.157189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:58.157189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:58.157189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:58.157189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:58.157189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 282/554] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:58.160578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8eee-7ba8-41d7-b60d-29951663cfa0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.160578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8eee-7ba8-41d7-b60d-29951663cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.160578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8eee-7ba8-41d7-b60d-29951663cfa0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.164342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 283/555] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:58.165249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e8eee-7ba8-41d7-b60d-29951663cfa0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.182957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.183541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.183823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.184159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.255802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:58.257306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:58.274149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.285492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.289164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e8eee-7ba8-41d7-b60d-29951663cfa0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.327307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598bda1-3946-4d1f-9029-829fabc918a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.328334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598bda1-3946-4d1f-9029-829fabc918a7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.328890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598bda1-3946-4d1f-9029-829fabc918a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.329303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598bda1-3946-4d1f-9029-829fabc918a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.330401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598bda1-3946-4d1f-9029-829fabc918a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:58.330910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598bda1-3946-4d1f-9029-829fabc918a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:58.332868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598bda1-3946-4d1f-9029-829fabc918a7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:58.346866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 284/556] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:58.348403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9be87c-06c3-4daa-aa21-e93d50fbde2c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.348887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9be87c-06c3-4daa-aa21-e93d50fbde2c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.349301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9be87c-06c3-4daa-aa21-e93d50fbde2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.366870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9b9be87c-06c3-4daa-aa21-e93d50fbde2c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:58.368263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9b9be87c-06c3-4daa-aa21-e93d50fbde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:58.375272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 273/557] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:05:58.375272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.459150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 285/558] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:58.460369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9be87c-06c3-4daa-aa21-e93d50fbde2c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.476024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14836ca-1b11-4f3f-94df-1ba999844473 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.498141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.514638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.515442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.515725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.516003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.523646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:58.525737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:05:58.541050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:58.542506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 274/559] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:58.543443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.583692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.605290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9be87c-06c3-4daa-aa21-e93d50fbde2c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.645975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bfd87-e605-4442-9fae-aca5cf9f8937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bfd87-e605-4442-9fae-aca5cf9f8937 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.647004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bfd87-e605-4442-9fae-aca5cf9f8937 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.647162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bfd87-e605-4442-9fae-aca5cf9f8937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.659125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a421c8-1f64-4205-9e6a-0dca7b417919 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.659757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bfd87-e605-4442-9fae-aca5cf9f8937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:58.661262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bfd87-e605-4442-9fae-aca5cf9f8937 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-331bfd87-e605-4442-9fae-aca5cf9f8937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:58.680788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:58.681988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:58.683381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:58.683381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:58.683381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:58.683381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:58.683381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 286/560] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:58.683878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1e1db-6d93-4269-ac02-e07de76f11ef None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.683878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1e1db-6d93-4269-ac02-e07de76f11ef None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.684703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1e1db-6d93-4269-ac02-e07de76f11ef None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.688247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 287/561] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:58.689318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1e1db-6d93-4269-ac02-e07de76f11ef None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.705960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.706755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.707034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.707405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.718764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:58.726746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:58.727895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 275/562] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:58.729585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.774623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.813350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1e1db-6d93-4269-ac02-e07de76f11ef None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.843997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa79477-a5f5-4853-8ee5-5fb44bec567a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:58.850374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901fd2f-96d2-46e3-9cc4-c687c5d81281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.851069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901fd2f-96d2-46e3-9cc4-c687c5d81281 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.851381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901fd2f-96d2-46e3-9cc4-c687c5d81281 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.851693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901fd2f-96d2-46e3-9cc4-c687c5d81281 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.852691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901fd2f-96d2-46e3-9cc4-c687c5d81281 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:58.853059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901fd2f-96d2-46e3-9cc4-c687c5d81281 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:58.854642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901fd2f-96d2-46e3-9cc4-c687c5d81281 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:58.856420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:58.878754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 288/563] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:58.886001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a81c-4b81-470e-8877-802b424d1945 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.886001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a81c-4b81-470e-8877-802b424d1945 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.886001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a81c-4b81-470e-8877-802b424d1945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.893739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:58.894407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:58.894738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:58.895019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:58.895763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:58.896070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:58.897465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:58.898335 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8760a81c-4b81-470e-8877-802b424d1945 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:58.899513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8760a81c-4b81-470e-8877-802b424d1945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:05:58.915645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 276/564] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:58.915645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:58.985006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 289/565] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:05:58.985006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760a81c-4b81-470e-8877-802b424d1945 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.028295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea0f1d-f0ae-44f9-a105-3b004523f284 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.030151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.077383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.078198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.078530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.078853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.079719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:59.080066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.082183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.098293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 277/566] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:59.098293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.110445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760a81c-4b81-470e-8877-802b424d1945 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.135917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.148769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.149556 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.149864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.150137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.160378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.162607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:59.185537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:59.186477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:59.187345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:59.187345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:59.187345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:59.187345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:59.187345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 290/567] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:59.187623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.213288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dc15d-a82d-4bba-939f-81939b57500f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.249526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.250245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.250534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.250814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.258034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.259505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:05:59.274691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:59.275747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 278/568] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:59.276770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.308508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.322134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575105f-4ea4-47b6-b2de-566da566234d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.357346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.358127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.358406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.358676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.371185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:05:59.381570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:59.382537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:59.383359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 291/569] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:59.384045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.395215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cf966-6e13-4f40-b85e-bbe468434750 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.425443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.435110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.435965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.436241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.436516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.447061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.448713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:05:59.470517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:59.471558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:59.472442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:05:59.472442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:05:59.472442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:05:59.472442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:05:59.476712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 279/570] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:59.477620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.512785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49d15-8025-4eeb-a73d-0df7a76c4609 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.548661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.549328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.549614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.549897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.550620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:59.550930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.552293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.562625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.564972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 292/571] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:59.566107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.592703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca06022-5e1a-48d1-a506-a5520aca39d3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.626674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.627440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.627773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.628088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.629002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:59.629303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.630956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.643195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 280/572] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:59.644039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.668137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.676479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee1767-7e2d-4ace-8e80-0793e2d41802 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.714071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818d6dc-c176-487c-8be1-dacdd858eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.714740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818d6dc-c176-487c-8be1-dacdd858eb34 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.715051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818d6dc-c176-487c-8be1-dacdd858eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.715436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818d6dc-c176-487c-8be1-dacdd858eb34 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.716204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818d6dc-c176-487c-8be1-dacdd858eb34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:59.716529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818d6dc-c176-487c-8be1-dacdd858eb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.717958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818d6dc-c176-487c-8be1-dacdd858eb34 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.726060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 293/573] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:59.727380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dfaff-cb35-4baa-98d9-11853bbc478c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.727667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dfaff-cb35-4baa-98d9-11853bbc478c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.728539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dfaff-cb35-4baa-98d9-11853bbc478c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.729900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 294/574] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:05:59.730593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dfaff-cb35-4baa-98d9-11853bbc478c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.759334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8ffe3-749d-4602-8845-665370882f85 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.793554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.794235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.794519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.794794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.795515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:59.795820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.797217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.799896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.811109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 281/575] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:59.811923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.846240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dfaff-cb35-4baa-98d9-11853bbc478c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.884709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.885679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.886027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.886363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.895372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.897215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:05:59.912672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:05:59.914024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 295/576] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:05:59.915352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:05:59.923891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2230280-0d58-4335-94cd-34bcc564a6e3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:05:59.934025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:05:59.957767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdbba2-7657-4f79-8828-9a264d879c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:05:59.958418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdbba2-7657-4f79-8828-9a264d879c60 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.958701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdbba2-7657-4f79-8828-9a264d879c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.958983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdbba2-7657-4f79-8828-9a264d879c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.959807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdbba2-7657-4f79-8828-9a264d879c60 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=cbaf3f85739348beb1bc8bae579f0da7) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:05:59.960112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdbba2-7657-4f79-8828-9a264d879c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:05:59.961592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdbba2-7657-4f79-8828-9a264d879c60 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:05:59.980582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 282/577] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/role_assignments?user.id=cbaf3f85739348beb1bc8bae579f0da7&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:05:59.980582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c92d7a-29b1-42ab-9984-c5ea21429add None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:05:59.980582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c92d7a-29b1-42ab-9984-c5ea21429add None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:05:59.980582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c92d7a-29b1-42ab-9984-c5ea21429add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:05:59.989390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f9c92d7a-29b1-42ab-9984-c5ea21429add None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:05:59.990607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f9c92d7a-29b1-42ab-9984-c5ea21429add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:00.045155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccf765-7ea9-4e3a-b469-9900e77a0be2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.076284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:00.093445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.094228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 283/578] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:00.095079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c92d7a-29b1-42ab-9984-c5ea21429add None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.105222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.106345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.106800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.122792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.124613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:00.146070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:00.147186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:00.148112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:00.148112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:00.148112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:00.148112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:00.148112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 296/579] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:00.148399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.209823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c92d7a-29b1-42ab-9984-c5ea21429add None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.234994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:00.250272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.251188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.251469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.251862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.260600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.262179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.271559 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268fc9b-83bf-47be-b6d7-cb696b7768af None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:00.279529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:00.280961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 284/580] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:00.282624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.315992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.316310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.316622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.316901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.317689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:00.317999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.319440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.338212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 297/581] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:00.338212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.345032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:00.410103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2966da-c5fe-442a-93da-f9583735737c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.449596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393cb3e-35d9-42c2-beeb-2360d0d1725f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.452136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.453084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.453366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.453643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.464845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:00.473202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:00.474164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:00.475036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 285/582] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:00.475721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.479430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:00.487985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.488917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.489081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.489350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.490070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:00.490375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.491794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.509931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 298/583] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:00.509931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.589473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febee145-e74a-4b57-bf78-9ced2cbf5671 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.601272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:00.629007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.629599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77ed66-6012-4a78-a88b-28cac1b6bfd8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.629727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.629953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.630235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.631071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:00.631376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.632824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.654244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 286/584] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:00.655399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.678038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.679987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.680437 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.681531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/48647bb993344a8b91cec7f623aea007/roles/9654f3986243401db4f1aad536273ede` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.706950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=48647bb993344a8b91cec7f623aea007, role_id=9654f3986243401db4f1aad536273ede)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.709334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.729391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 299/585] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:00 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/48647bb993344a8b91cec7f623aea007/roles/9654f3986243401db4f1aad536273ede => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:00.742597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.779456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f0fc5-3a73-4fb4-91ec-478068200c6f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.803863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:00.827594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.828388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.828775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.829144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.830359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:00.830749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.832538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.846822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 287/586] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:00.847886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.889471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd560fd4-a2c6-4b60-b65c-8c70b614a9e7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:00.894963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:00.933569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:00.934369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:00.934652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:00.934925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:00.946959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:00.949083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:00.971299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:00.972273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:00.973524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 300/587] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:00.974202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:00.997097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f7e7-67b3-4462-920d-ae1e3df115cc None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.037055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.037893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.038182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.038463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.053365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.054846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:01.075900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:01.077329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:01.078250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:01.078250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:01.078250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:01.078250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:01.080533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 288/588] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:01.081647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.088370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:01.094012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2adb-ed7f-40a4-b6f7-a075e462f714 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.132909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.133640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.133927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.134239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.145338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.146755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:01.170762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:01.171724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:01.172713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:01.172713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:01.172713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:01.172713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:01.172713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 301/589] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:01.176743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.183812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:01.204082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdfff-1d14-411c-9330-0bccdf6af233 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.242768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.243780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.244249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.244774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.245836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:01.246354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.248013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.267560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 289/590] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:01.275953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.317371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569b76d-f527-4505-9d1d-bc90c8b0966e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.364876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.365693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.366120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.366409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.367137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:01.367448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.369139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.386097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 302/591] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:01.386690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.389663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a6d96-644f-4f1a-b2ec-8417f97eda1a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.403253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:01.431403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.432481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.433341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.433822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.434747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:01.435257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.436989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.447755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:01.459331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 290/592] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:01.459331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.512870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502bead-f06a-4143-b3ba-a8c80631a52e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.551176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.552348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.552888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.553371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.565229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.566998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:01.585565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:01.586479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:01.588055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:01.588055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:01.588055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:01.588055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:01.588315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb27d-c35c-499b-90d6-d352de812f32 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.594921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 303/593] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:01.596119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.624271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.625570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.626095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.626620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.635067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.637066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:01.654600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:01.655726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:01.657064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:01.659659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 291/594] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:01.660927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.667763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:01.716301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9448a9-4e7c-4690-a2d8-dd07f9a47b28 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.731248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:01.753284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.754104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.754593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.754704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.763115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.764584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.779552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1ac40-3575-4dc2-beac-7a25137c0fb2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:01.779805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:01.780773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:06:01.781737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:01.784900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 304/595] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:01.786878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.823328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.824543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.825157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.825656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.826611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:01.827121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.828784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.842527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 292/596] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:01.843772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.898154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9aa61-e35c-43de-a3a6-fc646c5b74aa None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.925105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:01.939907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:01.940610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:01.940906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:01.941189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:01.941988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:01.942301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:01.943752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:01.957597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 305/597] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:01.958509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:01.975143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552487ab-0859-4ab3-a5be-a9ab542ce9f3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:01.998787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:02.015649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.016485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.016906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.017286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.018171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:02.018553 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.020069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.047523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 293/598] 10.222.0.52 () {56 vars in 1139 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:02.048461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.090036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74530f-f351-44bc-b288-a500f0f88976 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.131313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df31a92-d076-4e02-9c30-55bfe630e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.132106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df31a92-d076-4e02-9c30-55bfe630e51a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.132391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df31a92-d076-4e02-9c30-55bfe630e51a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.132714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df31a92-d076-4e02-9c30-55bfe630e51a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.140823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df31a92-d076-4e02-9c30-55bfe630e51a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.142265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df31a92-d076-4e02-9c30-55bfe630e51a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.154411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef7751-e665-4b2b-8e4c-e625c4860ba5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6df31a92-d076-4e02-9c30-55bfe630e51a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:02.156611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:02.157506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:02.158608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:02.162302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 306/599] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:02.163563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62900-8e86-4349-b5a6-cc7b79096aa9 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.163846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62900-8e86-4349-b5a6-cc7b79096aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.164130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62900-8e86-4349-b5a6-cc7b79096aa9 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.165488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 307/600] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:02.166180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a62900-8e86-4349-b5a6-cc7b79096aa9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.198172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.199087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.199366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.199637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/31bf0e50e20b47d0a63662bbecce833d/roles/9654f3986243401db4f1aad536273ede` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.215174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=31bf0e50e20b47d0a63662bbecce833d, role_id=9654f3986243401db4f1aad536273ede)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.216636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.218190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:02.230206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 294/601] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:02 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/31bf0e50e20b47d0a63662bbecce833d/roles/9654f3986243401db4f1aad536273ede => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:02.231199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.279712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:02.311682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a62900-8e86-4349-b5a6-cc7b79096aa9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.364266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfaa27-dbdb-4bf5-bcee-7662279fe9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.365628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfaa27-dbdb-4bf5-bcee-7662279fe9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.366060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfaa27-dbdb-4bf5-bcee-7662279fe9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.366754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfaa27-dbdb-4bf5-bcee-7662279fe9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.389413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0664924-d81b-426f-a070-ed97578716c1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.390764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfaa27-dbdb-4bf5-bcee-7662279fe9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.392925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfaa27-dbdb-4bf5-bcee-7662279fe9b2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-58dfaa27-dbdb-4bf5-bcee-7662279fe9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:02.412525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:02.413682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:02.414850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:02.414850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:02.414850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:02.414850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:02.417304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 308/602] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:02.418688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb797e-8a5c-45b7-880c-88c5e2aa018e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.419136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb797e-8a5c-45b7-880c-88c5e2aa018e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.419567 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb797e-8a5c-45b7-880c-88c5e2aa018e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.440908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.441145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-edcb797e-8a5c-45b7-880c-88c5e2aa018e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:02.441263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.441263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.441263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.441552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-edcb797e-8a5c-45b7-880c-88c5e2aa018e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:02.441640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:02.442232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.443475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.456286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 295/603] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:02.461443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.522599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 309/604] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:02.523901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb797e-8a5c-45b7-880c-88c5e2aa018e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.562926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:02.563449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:02.579577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb6d3-cf5d-44a8-8ec0-387454aafd22 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.616048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.617258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.617691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.618111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.619694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:02.620196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.621848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.640365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 296/605] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:02.641713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.644903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb797e-8a5c-45b7-880c-88c5e2aa018e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.683979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.684825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.685123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.685378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/28a07f47b8954418a5a86f344fb7fbe6/roles/9654f3986243401db4f1aad536273ede` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.701741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=28a07f47b8954418a5a86f344fb7fbe6, role_id=9654f3986243401db4f1aad536273ede)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.703195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.721036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 310/606] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:02 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/28a07f47b8954418a5a86f344fb7fbe6/roles/9654f3986243401db4f1aad536273ede => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:02.722604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.797966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cfb12-e4e9-4343-991d-c470a0e037ba None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.841329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:02.885711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c88bb-b46c-4acc-ac2f-d7be3da275d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.888026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c88bb-b46c-4acc-ac2f-d7be3da275d4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.888310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c88bb-b46c-4acc-ac2f-d7be3da275d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.888600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c88bb-b46c-4acc-ac2f-d7be3da275d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.897696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be247fd-07e3-4067-9c8b-04fcc0cebd11 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:02.907632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c88bb-b46c-4acc-ac2f-d7be3da275d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-362c88bb-b46c-4acc-ac2f-d7be3da275d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:02.916806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:02.917748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:02.918597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 297/607] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:02.920600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360d30-e361-4f2c-8ded-1d6895aa1be1 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.922432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360d30-e361-4f2c-8ded-1d6895aa1be1 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.922432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360d30-e361-4f2c-8ded-1d6895aa1be1 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.923563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 298/608] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:02.924526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a360d30-e361-4f2c-8ded-1d6895aa1be1 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:02.939281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b733f5-03a3-46d4-aa55-512807ceb9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:02.940669 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b733f5-03a3-46d4-aa55-512807ceb9da None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.941103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b733f5-03a3-46d4-aa55-512807ceb9da None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.941712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b733f5-03a3-46d4-aa55-512807ceb9da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:02.945919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:02.951339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b733f5-03a3-46d4-aa55-512807ceb9da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:02.953683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b733f5-03a3-46d4-aa55-512807ceb9da None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-47b733f5-03a3-46d4-aa55-512807ceb9da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:02.972053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:02.973081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:02.973970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:02.976701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 311/609] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:02.978605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c80e4-4f29-4a19-9269-be6cdd53c4cd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:02.979029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c80e4-4f29-4a19-9269-be6cdd53c4cd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:02.979436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c80e4-4f29-4a19-9269-be6cdd53c4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.000642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-076c80e4-4f29-4a19-9269-be6cdd53c4cd None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:03.002067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-076c80e4-4f29-4a19-9269-be6cdd53c4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:03.052805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a360d30-e361-4f2c-8ded-1d6895aa1be1 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.088439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 312/610] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:03.089882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c80e4-4f29-4a19-9269-be6cdd53c4cd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.093850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.094408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.094779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.095060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.095816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:03.096167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:03.097616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:03.107274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 299/611] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:03.113025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.177063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:03.188991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:03.217295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c80e4-4f29-4a19-9269-be6cdd53c4cd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.226093 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da50d35-f9d7-4e64-b5b4-fcedfa190390 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.266811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.267544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.267840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.268150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.268653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.268976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:03.269283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:03.270026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.270487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.270754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:03.271121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.283307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:03.285105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:03.286016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 313/612] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:03.286985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:03.304482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:03.305442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:03.306361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:03.306361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:03.306361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:03.306361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:03.308734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 300/613] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:03.309879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.413157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5277b6-b466-46ed-8b63-5d2145d6c3d2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.439701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4b4a9-b8e2-4acc-8696-a7a13d3de995 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.441466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:03.459008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.460316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.460671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.460993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.466909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:03.477487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:03.480169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.481326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.481818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.482342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/ff6a58944beb49519f991a753adaa521/roles/9654f3986243401db4f1aad536273ede` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:03.485570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:03.486739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:03.487811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:03.492651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 314/614] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:03.494101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.506524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=ff6a58944beb49519f991a753adaa521, role_id=9654f3986243401db4f1aad536273ede)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:03.508112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:03.523838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 301/615] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:03 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/ff6a58944beb49519f991a753adaa521/roles/9654f3986243401db4f1aad536273ede => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:03.524780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.637953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6bc9d-e021-40a8-aebb-6323fc2d33a5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.657275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0715e9-3fd8-45b5-ba23-5cedf989cb97 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.694458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.695457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.695751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.696035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.697382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:03.697748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:03.699152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:03.715038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 315/616] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:03.720588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.749466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.750128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.750409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.751491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.752271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:03.752592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:03.753967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:03.771193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 302/617] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:03.777190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.790781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:03.825326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:03.847818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3d9e-8e2b-42d5-917b-72d416dde5fe None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:03.883883 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:03.884769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:03.885058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:03.885327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:03.892855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:03.894604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:03.909538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:03.910824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:03.912070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:03.916598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 316/618] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:03.920544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:03.948936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266ef3-966b-4b53-95e1-1d5a8ced4188 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:04.023334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:04.024076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.024357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.024694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.051046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:04.052590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:04.083085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:04.084173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:04.085299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:04.085299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:04.085299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:04.085299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:04.094173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 303/619] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:04.099783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:04.117585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359213d-84c0-4207-b9d3-8725c0bd69e2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:04.123686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:04.186392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:04.187073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.187337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.187613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.188825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:04.189142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:04.190576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:04.222597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 317/620] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Jun 24 09:06:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:04.230244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:04.270125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:04.304685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afbc53-07b8-41c8-8dfd-704963879c4a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:04.372304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:04.373017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.373302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.373580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.374306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:04.374611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:04.375976 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:04.402515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 304/621] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:04.408937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:04.420819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39354d-6215-42d5-87d8-84152b7b2fbf None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:04.487882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:04.488767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.489052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.489331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.501663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:04.503127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:04.521657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:04.522619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:04.523517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:04.523517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:04.523517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:04.523517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:04.523517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 318/622] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:04.528316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:04.552924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:04.614133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae978-891d-468c-9ee8-cc56912d0160 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:04.669157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501fe6e-a1ed-476e-8e72-9aee90aea963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:04.670094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501fe6e-a1ed-476e-8e72-9aee90aea963 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.670377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501fe6e-a1ed-476e-8e72-9aee90aea963 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.670653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501fe6e-a1ed-476e-8e72-9aee90aea963 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.679557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501fe6e-a1ed-476e-8e72-9aee90aea963 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:04.681753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501fe6e-a1ed-476e-8e72-9aee90aea963 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:04.688275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6171d-4fc1-474e-9e77-fc7ee4e17bde None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-5501fe6e-a1ed-476e-8e72-9aee90aea963 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:04.700809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:04.701865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:04.702824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:04.703371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 305/623] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:06:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:04.705201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462202-3aef-4cd8-86cb-b2bb767e0fae None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.705489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462202-3aef-4cd8-86cb-b2bb767e0fae None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.707932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25462202-3aef-4cd8-86cb-b2bb767e0fae None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.707932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 306/624] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:04.712824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25462202-3aef-4cd8-86cb-b2bb767e0fae None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:04.726301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:04.730037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5a03f-6ce2-4be5-a6b9-8a7ac774631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:04.730967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5a03f-6ce2-4be5-a6b9-8a7ac774631d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.731376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5a03f-6ce2-4be5-a6b9-8a7ac774631d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.731851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5a03f-6ce2-4be5-a6b9-8a7ac774631d None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.732743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5a03f-6ce2-4be5-a6b9-8a7ac774631d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:04.733215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5a03f-6ce2-4be5-a6b9-8a7ac774631d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:04.736321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5a03f-6ce2-4be5-a6b9-8a7ac774631d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:04.749956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 319/625] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:04.756091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ae381-b6a5-4abd-a977-5047847b90e8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.756741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ae381-b6a5-4abd-a977-5047847b90e8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.757175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ae381-b6a5-4abd-a977-5047847b90e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.775983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b33ae381-b6a5-4abd-a977-5047847b90e8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:04.777555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b33ae381-b6a5-4abd-a977-5047847b90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:04.842855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25462202-3aef-4cd8-86cb-b2bb767e0fae None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:04.858263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 320/626] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:04.859494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ae381-b6a5-4abd-a977-5047847b90e8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:04.863030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:04.881644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:04.882933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:04.883407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:04.883844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:04.884964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:04.885439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:04.887993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:04.920743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 307/627] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:06:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:04.922831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:04.984407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ae381-b6a5-4abd-a977-5047847b90e8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.014190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:05.037447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.038523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.039028 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.039697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.049746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:05.051494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:05.067999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:05.069031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:05.069906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:05.076823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 321/628] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:05.076823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.077904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead60e5-3b84-4583-8563-9895dcb93604 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.114710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6bb1e-a815-4fba-a106-b8467b354b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.115510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6bb1e-a815-4fba-a106-b8467b354b65 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.115789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6bb1e-a815-4fba-a106-b8467b354b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.116061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6bb1e-a815-4fba-a106-b8467b354b65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.126362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6bb1e-a815-4fba-a106-b8467b354b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-63a6bb1e-a815-4fba-a106-b8467b354b65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:05.134134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:05.135063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:05.136610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 308/629] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:05.137341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755edfb4-f954-4049-9c38-20c6d1e72c43 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.137341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755edfb4-f954-4049-9c38-20c6d1e72c43 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.137341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755edfb4-f954-4049-9c38-20c6d1e72c43 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.143171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 309/630] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:05.143171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755edfb4-f954-4049-9c38-20c6d1e72c43 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.150380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:05.194461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8b43-fb09-42b0-a6ce-b028f83e4b0c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.231679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fc51c-42d9-4201-9060-3804013d898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.232356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc51c-42d9-4201-9060-3804013d898e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.232689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc51c-42d9-4201-9060-3804013d898e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.232890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc51c-42d9-4201-9060-3804013d898e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.253735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755edfb4-f954-4049-9c38-20c6d1e72c43 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.284175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:05.295726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.296549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.296968 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.297342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.298234 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:05.298627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:05.300129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:05.308634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fc51c-42d9-4201-9060-3804013d898e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:05.310846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fc51c-42d9-4201-9060-3804013d898e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:05.313174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 310/631] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:05.314145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.380736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:05.432469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 322/632] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:06:05.432469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331e738-f07e-4c38-bef0-529f46580b24 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.432469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331e738-f07e-4c38-bef0-529f46580b24 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.432469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331e738-f07e-4c38-bef0-529f46580b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.438722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a92893-f691-443c-bca2-b3e21a27c906 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.444581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8331e738-f07e-4c38-bef0-529f46580b24 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:05.445872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8331e738-f07e-4c38-bef0-529f46580b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:05.475692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.476357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.476680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.476956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.477749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:05.478041 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:05.479409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:05.488313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 311/633] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:05.489263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.521424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 323/634] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:05.522635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331e738-f07e-4c38-bef0-529f46580b24 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.549556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:05.611752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33cf9d-260b-4690-aa1b-cd6c9d7c81ad None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.632389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:05.641216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331e738-f07e-4c38-bef0-529f46580b24 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.649963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.650860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.651145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.651425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/cbaf3f85739348beb1bc8bae579f0da7/roles/9654f3986243401db4f1aad536273ede` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.666059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=cbaf3f85739348beb1bc8bae579f0da7, role_id=9654f3986243401db4f1aad536273ede)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:05.667500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:05.681346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.682226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.682910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.683205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.692840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 312/635] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:05 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/cbaf3f85739348beb1bc8bae579f0da7/roles/9654f3986243401db4f1aad536273ede => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:05.692840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.704444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:05.706276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:05.727152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:05.728331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:05.729443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:05.729443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:05.729443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:05.729443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:05.730377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 324/636] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:05.731309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.814315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:05.824422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3ff24-c6ef-417d-b9cb-a6af40fbd2e2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.868937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.869799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.870098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.870373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.872650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae584808-3e9b-47b5-b153-e2f12457893a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:05.881848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:05.883460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:05.902609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:05.903746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:05.905251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:05.905251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:05.905251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:05.905251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:05.905251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 313/637] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:05.905251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.921424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ed2f1-9fe8-4d2a-963b-f721cefcc66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:05.922623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed2f1-9fe8-4d2a-963b-f721cefcc66f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.922992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed2f1-9fe8-4d2a-963b-f721cefcc66f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.923248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ed2f1-9fe8-4d2a-963b-f721cefcc66f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.935527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ed2f1-9fe8-4d2a-963b-f721cefcc66f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ff6ed2f1-9fe8-4d2a-963b-f721cefcc66f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:05.943683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:05.944693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:05.945642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 325/638] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:05.946330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2ce64-646d-4d8c-a3cb-70fe33448f9b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:05.947321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2ce64-646d-4d8c-a3cb-70fe33448f9b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:05.947561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2ce64-646d-4d8c-a3cb-70fe33448f9b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:05.949200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 326/639] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:05.953730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2ce64-646d-4d8c-a3cb-70fe33448f9b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:05.959976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.020400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbe3eb-2175-4800-9016-58b42d029f70 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.070865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.071160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.071279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.071602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.072254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:06.072573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.073993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.080318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2ce64-646d-4d8c-a3cb-70fe33448f9b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.086116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 314/640] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:06.086950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.101042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.124152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50664dc1-e735-4ef0-a80e-438452adef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.124940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50664dc1-e735-4ef0-a80e-438452adef01 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.125224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50664dc1-e735-4ef0-a80e-438452adef01 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.125498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50664dc1-e735-4ef0-a80e-438452adef01 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.126322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50664dc1-e735-4ef0-a80e-438452adef01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:06.127403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50664dc1-e735-4ef0-a80e-438452adef01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.128089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50664dc1-e735-4ef0-a80e-438452adef01 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.142031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 327/641] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:06.143176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa833-7c08-456b-bec8-70e96f7c1db6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.143486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa833-7c08-456b-bec8-70e96f7c1db6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.143796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa833-7c08-456b-bec8-70e96f7c1db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.161390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e99fa833-7c08-456b-bec8-70e96f7c1db6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:06.162799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e99fa833-7c08-456b-bec8-70e96f7c1db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:06.198818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7359d-2790-4e83-a719-fc5cb859cedd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.231900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.238901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.239576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.239911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.240190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.248780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 328/642] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:06.248941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:06.249058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa833-7c08-456b-bec8-70e96f7c1db6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.249146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.249146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.258378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 315/643] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:06.259600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.340044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.355679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa833-7c08-456b-bec8-70e96f7c1db6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.375584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793c50a-d42a-4f0c-9277-a9efb45e906f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.391716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.392507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.392822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.393094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.400387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.401876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.413336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.414390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.414865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.415939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:06.416376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:06.417348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:06.418289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 329/644] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:06.418771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.432386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.434385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:06.449073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:06.450027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 316/645] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:06.451031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.490895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.540900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d9f31-1f97-47b1-a560-3b4bacb3bd1d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.564197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059f114-a5c0-4aca-b7bb-078616401b87 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.575647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.576431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.576748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.577027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.588791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.590875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.594677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.602685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.604259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.604546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.604851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:06.610468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:06.611758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:06.613451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:06.613451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:06.613451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:06.613451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:06.613451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 330/646] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:06.613451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.623004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:06.630989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:06.631898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:06.632787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 317/647] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:06.634266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.729077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ae607-78f2-4920-9249-55ac24db1e36 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.739235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.753331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d387c-bb8b-450e-b0a4-1ecfd5eafce7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.767663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.768713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.769138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.769414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.770229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:06.770499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.771916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.790700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 331/648] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:06.790700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.796138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.796867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.797154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.797432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.798655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:06.798972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.800802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.823840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 318/649] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:06.825173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.880575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:06.909812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125d491-d649-496e-a3f1-df53ed6b1991 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.940651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922f9b-f74b-4091-a675-e659943227f3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:06.946008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.947212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.947494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.947765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.948488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:06.948818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.950175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.962013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 332/650] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:06.967417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:06.978095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:06.978764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:06.979048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:06.979375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:06.980075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:06.980388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:06.981789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:06.990658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 319/651] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:06.991444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.003048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.085403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9ae55-42ba-40b3-9f93-527b6d02c20d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.114583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43683821-10b9-4f26-9a07-f66b7221c071 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.122364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.122785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.123116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.123439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.124294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=48647bb993344a8b91cec7f623aea007) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:07.124868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.126306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.126740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.140585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 333/652] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/role_assignments?user.id=48647bb993344a8b91cec7f623aea007&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:07.140585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.161016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.161783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.162142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.162428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.260365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.272241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bc28-d24d-4784-8ad6-5759a7d9ad35 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.278250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.280655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.314796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.316128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.316128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.316128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.328841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.329482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:07.350840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:07.351829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:06:07.352812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:07.355558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 334/653] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:07.356511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.431744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 320/654] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:06:07.433551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.456582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.505273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d312-cbe0-407f-b3fe-c7714c129004 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.541816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.542616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.542898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.543172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.550186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03d42-4e74-44f3-a56d-c52d9ec66a4d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.554536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.556632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:07.574582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:07.575645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:07.576671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:07.576671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:07.576671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:07.576671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:07.576671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 335/655] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:07.581539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.585365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.588701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.589824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.590258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.590680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.592452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:07.592951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.594710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.615427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 321/656] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:07.616602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.689778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.694525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ed1c8-484f-43d4-a4ac-2583de76e6f8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.729354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943fb0-7c35-496a-835f-9cc44d453ba2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.733306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.734309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.734716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.735092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.747607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.749679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.772280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:07.772521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:07.773586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:07.774703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:07.774703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:07.774703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:07.774703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:07.775695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.776122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.776535 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.777490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:07.777946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.779460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.781135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 336/657] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:07.782123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.795791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 322/658] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:07.797087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:07.864027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.910956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae07d4-2992-4c3d-98f5-ff657f700125 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.922971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6745041-70ff-4e3e-8ab3-5add3d01dfc8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:07.949687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:07.950022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c4caa-7418-44b2-bb01-838545e1646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.950682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c4caa-7418-44b2-bb01-838545e1646f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.950968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c4caa-7418-44b2-bb01-838545e1646f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.951254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c4caa-7418-44b2-bb01-838545e1646f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.959135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:07.959805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:07.960091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:07.960371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:07.961135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:07.961442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:07.963335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:07.975863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 323/659] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:07.981714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.033072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c4caa-7418-44b2-bb01-838545e1646f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.035230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c4caa-7418-44b2-bb01-838545e1646f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.095936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.099060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38d305-5dd3-45ab-b454-6949f11ebaca None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.135772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.136594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.136883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.137159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.145266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.147017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.155249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 337/660] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:06:08.156860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20504de-b53d-4f6c-9211-9ae3bbabbcd6 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.157289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20504de-b53d-4f6c-9211-9ae3bbabbcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.157722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20504de-b53d-4f6c-9211-9ae3bbabbcd6 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.159175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 338/661] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:08.160464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20504de-b53d-4f6c-9211-9ae3bbabbcd6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:08.169893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:08.170930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 324/662] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:08.172542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.226812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.287276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee36a3-8a85-4306-ae9a-dccb1d72c2f3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.289978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20504de-b53d-4f6c-9211-9ae3bbabbcd6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.326648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428692f-a61e-445e-8d82-59c2759d6f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.326846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.327433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428692f-a61e-445e-8d82-59c2759d6f03 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.327549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.327795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.327895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428692f-a61e-445e-8d82-59c2759d6f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.328064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428692f-a61e-445e-8d82-59c2759d6f03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.328171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.335319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428692f-a61e-445e-8d82-59c2759d6f03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.336811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428692f-a61e-445e-8d82-59c2759d6f03 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.339282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.340745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-b428692f-a61e-445e-8d82-59c2759d6f03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:08.351123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:08.352128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:08.353637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:08.357221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 339/663] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:08.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:08.359012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:08.361038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:08.361038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:08.361038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:08.361038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:08.361560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3788d-ce4b-40f8-9b78-bf5743e3bf01 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.361560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3788d-ce4b-40f8-9b78-bf5743e3bf01 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.361560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3788d-ce4b-40f8-9b78-bf5743e3bf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.361977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 325/664] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:08.361977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.382000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bdf3788d-ce4b-40f8-9b78-bf5743e3bf01 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:08.383298 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bdf3788d-ce4b-40f8-9b78-bf5743e3bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:08.396487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.464606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 340/665] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:08.464606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3788d-ce4b-40f8-9b78-bf5743e3bf01 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.482204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.485916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fc9a-5664-471d-b139-628e6d8a64ef None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.524446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.525175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.525462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.525737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.526573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:08.526884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.528324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.545830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 326/666] 10.222.0.52 () {56 vars in 1139 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:08.546630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.584484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3788d-ce4b-40f8-9b78-bf5743e3bf01 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.625002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.626190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.626630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.627044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.628108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:08.628601 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.630262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.648647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 341/667] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:08.650212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.654464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38267f8-3f97-40d6-9b21-4e4a573013c5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.659215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.691127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.691806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.692089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.692368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.693121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:08.693429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.694829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.710303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 327/668] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:08.710303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.731714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.765389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54295f6c-3270-4cc4-a707-e18c583d7265 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.802844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.803648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.803941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.804221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.817023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:08.826419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568c757-4a0c-42b2-b7ec-04af4b735b62 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:08.826964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:08.828413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:08.831801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:08.833694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 342/669] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:08.834706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:08.867688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.868347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.868685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.869216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.908166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.945532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.946989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:08.947742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e5c71-3d0b-440d-870d-2a6770062513 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:08.966149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:08.988299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:08.989417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:08.989715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:08.989998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:08.990838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=31bf0e50e20b47d0a63662bbecce833d) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:08.991148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:08.992989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.008594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 343/670] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/role_assignments?user.id=31bf0e50e20b47d0a63662bbecce833d&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:09.008594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.101829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 328/671] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:06:09.102064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.144354 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495f331-d2d3-4840-994d-7d3de5b13cd5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.183567 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:09.191726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.192531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.192841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.193117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.204835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.207125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:09.225571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:09.226540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 344/672] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:09.228859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.254342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58557d4a-2c65-47af-befd-e538fdc932d2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.276487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:09.297456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.298386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.298726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.299101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.300354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:09.301031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.302761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.318152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 329/673] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:09.318152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.361341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3e66-90b4-43a2-bbf0-07a6b605580c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.398187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.398991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.399277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.400004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.411184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.412674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.429409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd83774-c1c3-48b0-9fa0-9047af5d6595 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:09.432081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:09.433334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:09.434378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:09.434378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:09.434378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:09.434378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:09.434378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 345/674] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:09.434378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.470493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d936-ffde-4cdf-927f-f2726e3b9cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.471155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d936-ffde-4cdf-927f-f2726e3b9cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.471429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d936-ffde-4cdf-927f-f2726e3b9cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.471883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d936-ffde-4cdf-927f-f2726e3b9cf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.472470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d936-ffde-4cdf-927f-f2726e3b9cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:09.472870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d936-ffde-4cdf-927f-f2726e3b9cf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.474219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d936-ffde-4cdf-927f-f2726e3b9cf7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.485538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:09.492458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 330/675] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:09.492458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8240cca-cff9-402e-af71-062565a19314 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.492458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8240cca-cff9-402e-af71-062565a19314 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.492458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8240cca-cff9-402e-af71-062565a19314 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.493183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 331/676] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:09.494594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8240cca-cff9-402e-af71-062565a19314 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.541151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:09.556573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367b372-d475-491c-ad61-6d2881ddb462 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.594754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.595662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.595941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.596212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.608292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.609638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.610524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8240cca-cff9-402e-af71-062565a19314 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:09.628219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:09.629206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:09.630100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:09.630100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:09.630100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:09.630100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:09.630100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 346/677] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:09.630385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.656591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15483260-b162-4fb0-aa8a-2b882862b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.657370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15483260-b162-4fb0-aa8a-2b882862b776 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.657720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15483260-b162-4fb0-aa8a-2b882862b776 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.658063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15483260-b162-4fb0-aa8a-2b882862b776 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.658842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15483260-b162-4fb0-aa8a-2b882862b776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:09.659205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15483260-b162-4fb0-aa8a-2b882862b776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.660657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15483260-b162-4fb0-aa8a-2b882862b776 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.676774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 332/678] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:09.676774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad340250-a808-4dfa-ad05-65885efc64ac None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.676774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad340250-a808-4dfa-ad05-65885efc64ac None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.676774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad340250-a808-4dfa-ad05-65885efc64ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.691965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ad340250-a808-4dfa-ad05-65885efc64ac None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:09.693094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ad340250-a808-4dfa-ad05-65885efc64ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:09.741580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:09.749755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444151-8847-4d16-b43b-b985f79b6d34 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.767539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 333/679] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:09.768389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad340250-a808-4dfa-ad05-65885efc64ac None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.802928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:09.803146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.803683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.803993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.804300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.805284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=28a07f47b8954418a5a86f344fb7fbe6) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:09.805676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.808054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.824585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 347/680] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/role_assignments?user.id=28a07f47b8954418a5a86f344fb7fbe6&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:09.824585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.904061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad340250-a808-4dfa-ad05-65885efc64ac None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.949378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b93473-a079-4a27-9431-1107f449185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.950550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b93473-a079-4a27-9431-1107f449185f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.950948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b93473-a079-4a27-9431-1107f449185f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.951230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b93473-a079-4a27-9431-1107f449185f None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:09.952227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b93473-a079-4a27-9431-1107f449185f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:09.952538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b93473-a079-4a27-9431-1107f449185f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:09.954734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b93473-a079-4a27-9431-1107f449185f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:09.956256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f26e0d-4279-48b8-af58-8723ca5b225c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:09.971181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 334/681] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:09.972333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b93473-a079-4a27-9431-1107f449185f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:09.996791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:09.997127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:09.997533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:09.998496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.006845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.008362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:10.022991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:10.023916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:10.024949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 348/682] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:10.025478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.052492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.095004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.110757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b93473-a079-4a27-9431-1107f449185f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.153245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.154655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.154941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.155219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.162378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd047a-dcf0-4387-a3a8-8a43e54fd469 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.169006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:10.179454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:10.180512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 335/683] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:10.181886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.209345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.210256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.210619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.210900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.221114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.222779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:10.241201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:10.242229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 349/684] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:10.243640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.305355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25034f46-53c1-4dea-ada9-129929629fa3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.331184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.340274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.340972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.341253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.341575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.342430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:10.342958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.344517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.357151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2506ab-4be5-4526-a97a-76fcdfef0561 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.363138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 336/685] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:10.364109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.376392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.396358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.397691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.398121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.398540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.400242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:10.400768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.402329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.417141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 350/686] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:10.418259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.470937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a04101-84ab-4f7e-9ada-3aa9838f2002 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.507661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.508296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.508448 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.508798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.510250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:10.510539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.512131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.533431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72f13e-d65e-4dd4-8bc7-664de499e22d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.536942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 337/687] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:10.537874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.572733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.573726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.574174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.574649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.575768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=ff6a58944beb49519f991a753adaa521) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:10.576237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.577882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.585040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 351/688] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/role_assignments?user.id=ff6a58944beb49519f991a753adaa521&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:10.586121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.590620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.634851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.699565 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804cf9f-0fb6-4df1-bb3f-7a373deed309 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.705360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6995f74-3c59-42df-8e0f-480dea11d593 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.745557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.746359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.746639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.746969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.758181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.760197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.777249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.778380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.778819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.779234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:10.780759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:10.781804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:10.782837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:10.782837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:10.782837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:10.782837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:10.782837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 338/689] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:10.791851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:10.920888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee85ba-9a5a-4c67-b95b-7aa4bc91e5a8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:10.925214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.926685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.933902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.951222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:10.970475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:10.971398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:10.971685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:10.971959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:10.972735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:10.973042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:10.974410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:10.990579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 339/690] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:10.995971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.074082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 352/691] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:06:11.079501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.127171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccca00b-3e58-4820-adbc-a9e4ec830c26 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:11.166033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:11.167216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.167728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.168211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.177684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:11.179497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:11.195193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:11.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:11.197288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:11.199934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 340/692] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:06:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:11.205631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.253501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:11.257019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516537e0-f167-441a-844d-6ffeb553e794 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:11.260804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:11.307089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8139db-40d5-4af5-a06a-20c7a3e96e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:11.309153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8139db-40d5-4af5-a06a-20c7a3e96e54 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.309886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8139db-40d5-4af5-a06a-20c7a3e96e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.310392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8139db-40d5-4af5-a06a-20c7a3e96e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.319272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8fbc2-4a90-4e07-bf42-ba5190c765ee None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:11.324134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8139db-40d5-4af5-a06a-20c7a3e96e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:11.325624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8139db-40d5-4af5-a06a-20c7a3e96e54 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-4f8139db-40d5-4af5-a06a-20c7a3e96e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:11.344152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:11.345140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:11.346030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:11.346030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:11.346030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:11.346030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:11.346030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 353/693] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:11.347259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7115f-fdd5-4b79-a0b8-1de2a37e1ac3 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.347546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7115f-fdd5-4b79-a0b8-1de2a37e1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.347821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7115f-fdd5-4b79-a0b8-1de2a37e1ac3 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.353214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 354/694] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:11.354550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7115f-fdd5-4b79-a0b8-1de2a37e1ac3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.361375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82e5d9-66ff-45a2-be91-bc0bf8ffd0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:11.362337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e5d9-66ff-45a2-be91-bc0bf8ffd0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.362843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e5d9-66ff-45a2-be91-bc0bf8ffd0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.363360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e5d9-66ff-45a2-be91-bc0bf8ffd0c6 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.364518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e5d9-66ff-45a2-be91-bc0bf8ffd0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:11.365056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e5d9-66ff-45a2-be91-bc0bf8ffd0c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:11.366805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e5d9-66ff-45a2-be91-bc0bf8ffd0c6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:11.387256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 341/695] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:11.392092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1f817-ac6c-4244-a1a0-00a8d8f2eb0b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.392092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1f817-ac6c-4244-a1a0-00a8d8f2eb0b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.392092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1f817-ac6c-4244-a1a0-00a8d8f2eb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.410650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-98d1f817-ac6c-4244-a1a0-00a8d8f2eb0b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:11.411830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-98d1f817-ac6c-4244-a1a0-00a8d8f2eb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:11.486353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7115f-fdd5-4b79-a0b8-1de2a37e1ac3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:11.488828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 342/696] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:11.489770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1f817-ac6c-4244-a1a0-00a8d8f2eb0b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.537560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:11.539063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:11.539669 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.539944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.541098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.541866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:11.542086 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:11.543458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:11.556857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 355/697] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:11.558022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.588986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:11.641203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1f817-ac6c-4244-a1a0-00a8d8f2eb0b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:11.676869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a62d66-e4a8-476e-aef7-38adf7ebe3e2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:11.685389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:11.686141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.686500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.686860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.726159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:11.727310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.727803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.728743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.729446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:11.732648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:11.748174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:11.749220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:11.750182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:11.755749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 356/698] 10.222.0.52 () {56 vars in 1129 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:11.756807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.769995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:11.772533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:11.810954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:11.846922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:11.875922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061f6f2-49ce-4048-9472-b0b94489d398 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:11.912425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:11.913269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:11.913557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:11.913838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:11.917487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 343/699] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:06:11.918616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:11.928324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:11.928324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:11.947663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:11.948617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:11.949511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:11.950547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 357/700] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:11.951637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.054284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fafc2-4a78-473c-a0ed-b1199561e1ca None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.073703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05593ea-dc56-42c8-bf96-4f92888ab35c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.084349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:12.095647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.096324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.096630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.096911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.097784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:12.098092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.099754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.110156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 344/701] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:12.110156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.122393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.123538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.124043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.124526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.135600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.137428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.138414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:12.159785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:12.160783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:12.161663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:12.161663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:12.161663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:12.161663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:12.163964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 358/702] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:12.165662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.227235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e11fc6-4b88-4723-9130-6051153c8320 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.264508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.265224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.265523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.265796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.266588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:12.266900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.268330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.271476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e26e0-cec3-4b80-b535-4b465c7b7b37 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.280052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 345/703] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:12.280892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.316048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36509e5-f28d-4001-a967-676bb01949f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.317266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36509e5-f28d-4001-a967-676bb01949f1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.317715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36509e5-f28d-4001-a967-676bb01949f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.318312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36509e5-f28d-4001-a967-676bb01949f1 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.319817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36509e5-f28d-4001-a967-676bb01949f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:12.320329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36509e5-f28d-4001-a967-676bb01949f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.322691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36509e5-f28d-4001-a967-676bb01949f1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.338151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 359/704] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:12.338151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497725ce-b03e-449f-9bca-065b9f0927cf None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.338151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497725ce-b03e-449f-9bca-065b9f0927cf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.338151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497725ce-b03e-449f-9bca-065b9f0927cf None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.338151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 360/705] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:12.338151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497725ce-b03e-449f-9bca-065b9f0927cf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.354634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:12.413572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fdeaf-a252-47fc-9f89-c3c4d7ccd8d7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.419356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:12.463750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce25da4-6535-4638-b27a-b846ea7bec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.464987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce25da4-6535-4638-b27a-b846ea7bec28 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.465280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce25da4-6535-4638-b27a-b846ea7bec28 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.465854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce25da4-6535-4638-b27a-b846ea7bec28 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.466634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce25da4-6535-4638-b27a-b846ea7bec28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:12.466950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce25da4-6535-4638-b27a-b846ea7bec28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.469122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce25da4-6535-4638-b27a-b846ea7bec28 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.478858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497725ce-b03e-449f-9bca-065b9f0927cf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.487269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 346/706] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:12.489194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a00b2-01f2-4215-9bc1-a9bcd5bdbd64 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.489194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a00b2-01f2-4215-9bc1-a9bcd5bdbd64 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.489194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a00b2-01f2-4215-9bc1-a9bcd5bdbd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.509827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-725a00b2-01f2-4215-9bc1-a9bcd5bdbd64 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:12.511191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-725a00b2-01f2-4215-9bc1-a9bcd5bdbd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:12.527082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.528406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.528974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.529323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.531790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=cbaf3f85739348beb1bc8bae579f0da7) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:12.531790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.532211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.538760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 361/707] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/role_assignments?user.id=cbaf3f85739348beb1bc8bae579f0da7&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:12.539730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.647019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 347/708] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:12.683581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a00b2-01f2-4215-9bc1-a9bcd5bdbd64 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.738037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5f98d-dd69-4c67-b6ac-ca6f75738770 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.759644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:12.778344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:12.809357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.809574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.809913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.810348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.818562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.820117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:12.838076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:12.839093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:12.840061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 362/709] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:12.847006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.851277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a00b2-01f2-4215-9bc1-a9bcd5bdbd64 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:12.892187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:12.893069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:12.893356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:12.893634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:12.908928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:12.910989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:12.934732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:12.935735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:12.937095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:12.937095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:12.937095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:12.937095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:12.944214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 348/710] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:12.945486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:12.969564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e9912-76d2-47f3-9575-de6bced1fd42 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:13.006329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:13.007198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.007311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.007526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.008317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:13.008654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:13.010856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:13.027114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 363/711] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:13.038220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:13.066992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:13.079147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9b50b-3b70-435f-839b-f9fd1d887cfe None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:13.088077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:13.124320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51498e34-b3c5-464a-9d9d-6421ab114d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:13.125372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51498e34-b3c5-464a-9d9d-6421ab114d49 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.125844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51498e34-b3c5-464a-9d9d-6421ab114d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.126308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51498e34-b3c5-464a-9d9d-6421ab114d49 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.127220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51498e34-b3c5-464a-9d9d-6421ab114d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:13.127730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51498e34-b3c5-464a-9d9d-6421ab114d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:13.129331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51498e34-b3c5-464a-9d9d-6421ab114d49 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:13.144298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 349/712] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:13.145999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab9bd3-319b-4ffb-9652-ab9914c87413 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.146295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab9bd3-319b-4ffb-9652-ab9914c87413 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.146599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab9bd3-319b-4ffb-9652-ab9914c87413 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.147990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 350/713] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:13.150634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab9bd3-319b-4ffb-9652-ab9914c87413 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:13.162895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754eaab8-ca63-4d8d-a68c-f061b3d444e0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:13.203337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006bbca-6b8d-4b7e-9a13-96f25cbc2bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:13.203972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006bbca-6b8d-4b7e-9a13-96f25cbc2bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.204261 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006bbca-6b8d-4b7e-9a13-96f25cbc2bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.204542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006bbca-6b8d-4b7e-9a13-96f25cbc2bd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.205408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006bbca-6b8d-4b7e-9a13-96f25cbc2bd7 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=48647bb993344a8b91cec7f623aea007) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:13.205711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006bbca-6b8d-4b7e-9a13-96f25cbc2bd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:13.207157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006bbca-6b8d-4b7e-9a13-96f25cbc2bd7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:13.213224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 364/714] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:13 2022] GET /identity/v3/role_assignments?user.id=48647bb993344a8b91cec7f623aea007&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:13.214400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a067c-5116-4d2b-9cc6-c06671acaec0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.214690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a067c-5116-4d2b-9cc6-c06671acaec0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.214969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a067c-5116-4d2b-9cc6-c06671acaec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.244597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c24a067c-5116-4d2b-9cc6-c06671acaec0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:13.245861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c24a067c-5116-4d2b-9cc6-c06671acaec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:13.277574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab9bd3-319b-4ffb-9652-ab9914c87413 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:13.318634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:13.319722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.320227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.320344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 365/715] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:13.320862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.322427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:13.324133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:13.347914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a067c-5116-4d2b-9cc6-c06671acaec0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:13.364453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 351/716] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:06:13 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:13.405368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:13.498030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a067c-5116-4d2b-9cc6-c06671acaec0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:13.545352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:13.546951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.547480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.547950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] PATH_INFO: `/v3/services/network` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.548831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:13.550432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:13.564238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:13.565448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:13.566297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:13.569219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 366/717] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:13 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:13.575505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:13.592734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:13.704618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4789ba-a35b-43aa-9c15-799c63762340 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:13.744724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461809c-2ad1-44f1-9759-6b2ad2a89b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:13.745405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461809c-2ad1-44f1-9759-6b2ad2a89b5d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.745695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461809c-2ad1-44f1-9759-6b2ad2a89b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.745973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461809c-2ad1-44f1-9759-6b2ad2a89b5d None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.746697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461809c-2ad1-44f1-9759-6b2ad2a89b5d None admin] RBAC: Adding query filter params (name=network) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:13.747007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461809c-2ad1-44f1-9759-6b2ad2a89b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:13.748442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461809c-2ad1-44f1-9759-6b2ad2a89b5d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:13.753564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 352/718] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:13.759656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:13.871512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:13.876322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50ad5b-698e-4d1e-9e56-e3930c3e136f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:13.914021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48019a4-3268-4ca8-a282-f1cd4246ad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:13.914684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48019a4-3268-4ca8-a282-f1cd4246ad27 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:13.914964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48019a4-3268-4ca8-a282-f1cd4246ad27 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:13.915803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48019a4-3268-4ca8-a282-f1cd4246ad27 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:13.916510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48019a4-3268-4ca8-a282-f1cd4246ad27 None admin] RBAC: Adding query filter params (type=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:13.916841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48019a4-3268-4ca8-a282-f1cd4246ad27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:13.918259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48019a4-3268-4ca8-a282-f1cd4246ad27 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:13.923273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 367/719] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:14.662424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cefdf1-d4c0-49d9-b60f-3bbfe8b0d6ca None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:14.662811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cefdf1-d4c0-49d9-b60f-3bbfe8b0d6ca None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:14.662997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cefdf1-d4c0-49d9-b60f-3bbfe8b0d6ca None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:14.664589 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 353/720] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:14.671003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f2262-a4c0-48a0-84fd-f77b0583182d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:14.671527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f2262-a4c0-48a0-84fd-f77b0583182d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:14.671754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f2262-a4c0-48a0-84fd-f77b0583182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:14.690325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-722f2262-a4c0-48a0-84fd-f77b0583182d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:14.691497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-722f2262-a4c0-48a0-84fd-f77b0583182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:14.771463 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 368/721] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:06:14.806734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cefdf1-d4c0-49d9-b60f-3bbfe8b0d6ca None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:14.909109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:14.935933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cefdf1-d4c0-49d9-b60f-3bbfe8b0d6ca None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:14.970775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252c1c7-b10e-4992-beb4-f854499e7f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:14.971442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c1c7-b10e-4992-beb4-f854499e7f8f None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:14.971722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c1c7-b10e-4992-beb4-f854499e7f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:14.972000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c1c7-b10e-4992-beb4-f854499e7f8f None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:14.972752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252c1c7-b10e-4992-beb4-f854499e7f8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:14.974110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252c1c7-b10e-4992-beb4-f854499e7f8f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:15.001698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 354/722] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:14 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:15.408319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189da9b9-2cc7-423b-a072-a21b3f016f38 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:15.408725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189da9b9-2cc7-423b-a072-a21b3f016f38 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:15.408906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189da9b9-2cc7-423b-a072-a21b3f016f38 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:15.410525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 369/723] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:15.416616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8712bce-c003-473d-a311-8cb9e448a5b3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:15.416908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8712bce-c003-473d-a311-8cb9e448a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:15.417186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8712bce-c003-473d-a311-8cb9e448a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:15.436133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e8712bce-c003-473d-a311-8cb9e448a5b3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:15.437423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e8712bce-c003-473d-a311-8cb9e448a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:15.521233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 355/724] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:15.548084 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189da9b9-2cc7-423b-a072-a21b3f016f38 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:15.684748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:15.697468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189da9b9-2cc7-423b-a072-a21b3f016f38 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:15.740160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d28e7c-c9cb-41c3-ac87-61f6edcb8b81 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:15.740160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d28e7c-c9cb-41c3-ac87-61f6edcb8b81 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:15.740160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d28e7c-c9cb-41c3-ac87-61f6edcb8b81 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:15.740160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 356/725] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:15.745949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5244-fb05-4def-abf0-eaca9ce59723 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:15.746484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5244-fb05-4def-abf0-eaca9ce59723 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:15.746990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5244-fb05-4def-abf0-eaca9ce59723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:15.748342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:15.753239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:15.754091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:15.755091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:15.780019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:15.781680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-201d5244-fb05-4def-abf0-eaca9ce59723 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:15.782890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-201d5244-fb05-4def-abf0-eaca9ce59723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:15.788832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:15.789761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:15.790624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:15.800322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 370/726] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:15.801923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:15.905549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:15.944784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 357/727] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:06:15.957605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf5dd8-c093-4e34-96a3-dd8a5e325485 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:15.987601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5244-fb05-4def-abf0-eaca9ce59723 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:15.994327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:15.995258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:15.995746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:15.996358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:15.997339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:15.997945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:15.999507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:16.009266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 371/728] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:16.016402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.056941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:16.179969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5244-fb05-4def-abf0-eaca9ce59723 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.219597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f77fa-e310-4c24-b99b-af41aa030dc1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.255179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446527b3-80af-4881-8450-f040918ff25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.255986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446527b3-80af-4881-8450-f040918ff25c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.256270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446527b3-80af-4881-8450-f040918ff25c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.256546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446527b3-80af-4881-8450-f040918ff25c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.281130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446527b3-80af-4881-8450-f040918ff25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-446527b3-80af-4881-8450-f040918ff25c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:16.288818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:16.290393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:16.291262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 358/729] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:16.292269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533d72-f6dd-48e7-ac7d-8bd214be556a None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.292451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533d72-f6dd-48e7-ac7d-8bd214be556a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.293626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533d72-f6dd-48e7-ac7d-8bd214be556a None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.295171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 359/730] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:16.299513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d8e85-ea93-4b45-8afa-144e7e892481 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.299902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d8e85-ea93-4b45-8afa-144e7e892481 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.300185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d8e85-ea93-4b45-8afa-144e7e892481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.302273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.303144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.303492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.303829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.316540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:16.318158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:16.320042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ae6d8e85-ea93-4b45-8afa-144e7e892481 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:16.322266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ae6d8e85-ea93-4b45-8afa-144e7e892481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:16.337239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:16.339069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:16.341014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:16.341014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:16.341014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:16.341014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:16.341014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 372/731] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:16.341579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.400975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:16.416879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 360/732] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:06:16.417772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d8e85-ea93-4b45-8afa-144e7e892481 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.419627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:16.463684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdce1b-0693-45d3-88b5-1653b9471116 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.499136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.499730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.500012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.500384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.501262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:16.501570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:16.502930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:16.512577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 373/733] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:16.514763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.542834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d8e85-ea93-4b45-8afa-144e7e892481 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.578464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.579141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.579425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.579702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.580428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:16.580762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:16.582131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:16.594606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 361/734] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:16.595426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.627809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ddef7-c56d-46bc-b2d6-2bb445be9ecb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.643631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:16.655591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:16.664883 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaae89-353a-49a8-8848-792a0117e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.666364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaae89-353a-49a8-8848-792a0117e570 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.666648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaae89-353a-49a8-8848-792a0117e570 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.666917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaae89-353a-49a8-8848-792a0117e570 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.679323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaae89-353a-49a8-8848-792a0117e570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-facaae89-353a-49a8-8848-792a0117e570 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:16.687989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:16.689406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:16.690280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 374/735] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:16.690979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca858b2-76ee-4062-bb43-8a1abb9c3753 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.691055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca858b2-76ee-4062-bb43-8a1abb9c3753 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.691698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca858b2-76ee-4062-bb43-8a1abb9c3753 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.696863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 375/736] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:16.696863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca858b2-76ee-4062-bb43-8a1abb9c3753 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.724396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c9b3-30e3-4e55-a5a5-5d23813175d2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.761169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.761976 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.762260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.762543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.773871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:16.775341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:16.793981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:16.794941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:16.795851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:16.795851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:16.795851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:16.795851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:16.795851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 362/737] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:16.796137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.810034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca858b2-76ee-4062-bb43-8a1abb9c3753 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.847718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873566e-fc64-4689-a138-fe5d6039bba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.848547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873566e-fc64-4689-a138-fe5d6039bba4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.848861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873566e-fc64-4689-a138-fe5d6039bba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.849143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873566e-fc64-4689-a138-fe5d6039bba4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.857598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873566e-fc64-4689-a138-fe5d6039bba4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:16.859035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873566e-fc64-4689-a138-fe5d6039bba4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9873566e-fc64-4689-a138-fe5d6039bba4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:16.873030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:16.873967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:16.874881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 376/738] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:16.875462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85a149-5dea-4c98-a57d-1ace83519f7e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.876436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85a149-5dea-4c98-a57d-1ace83519f7e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.876436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85a149-5dea-4c98-a57d-1ace83519f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.892099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ef85a149-5dea-4c98-a57d-1ace83519f7e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:16.893337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ef85a149-5dea-4c98-a57d-1ace83519f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:16.905965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:16.915641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc88a7-2e7d-435a-b0a8-06e9eddf2737 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:16.942469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:16.952367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:16.953069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:16.953354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:16.953629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:16.954369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:16.954675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:16.956040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:16.964595 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 363/739] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:16.965346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:16.974891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 377/740] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:06:16.975603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85a149-5dea-4c98-a57d-1ace83519f7e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.085146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eceec8a-7b08-4bf5-b165-361868dd72a5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.085997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85a149-5dea-4c98-a57d-1ace83519f7e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.120722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.121395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.121683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.121964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.122682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:17.122994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.124372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.124647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.125319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.125594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.125895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.126995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:17.126995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.128397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.136959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 364/741] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:17.137622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.150963 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 378/742] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Jun 24 09:06:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:17.151770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.157935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:17.190661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:17.248364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de31953-0de0-4042-89e4-1ddb3dc42ac6 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.270488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b1f20-19a4-4b95-9919-37074205d8c0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.285165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.285972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.286340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.286631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.297299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.298691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.308519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.309356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.309661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.309945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.310668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.312033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:17.316879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:17.317929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:17.318885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:17.318885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:17.318885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:17.318885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:17.318885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 365/743] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:17.319159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.331491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 379/744] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:17 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:17.332548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.422059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:17.459890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:17.470482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3a725-36dd-43e0-af5c-99cbb5b09fed None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.478488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f27ad1-2ec7-4458-873c-89de46fcc7be None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.520575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.521432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.521721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.521999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.531131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f327a4d-c64c-4aa0-b856-0be22505c664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.531723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.531833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f327a4d-c64c-4aa0-b856-0be22505c664 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.532061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f327a4d-c64c-4aa0-b856-0be22505c664 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.532338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f327a4d-c64c-4aa0-b856-0be22505c664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.533254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:17.548643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:17.549592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 366/745] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:06:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:17.550528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.621857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f327a4d-c64c-4aa0-b856-0be22505c664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.623318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f327a4d-c64c-4aa0-b856-0be22505c664 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.664717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b51f8a-9094-4ea7-9b3b-642d1ff92fea None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.693064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:17.703182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.703849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.704129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.704404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.705151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:17.705461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.706937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.719038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 367/746] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:17.719860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.728336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:17.744611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 380/747] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:06:17.744611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f79b8-c392-4f0a-ad02-2f509a42bfe0 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.744611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f79b8-c392-4f0a-ad02-2f509a42bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.744611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f79b8-c392-4f0a-ad02-2f509a42bfe0 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.744611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 381/748] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:17.744611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f79b8-c392-4f0a-ad02-2f509a42bfe0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.836514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951186b-3013-49ad-b73c-d7396da27815 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.859363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f79b8-c392-4f0a-ad02-2f509a42bfe0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:17.874737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.875494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.875782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.876061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.877056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:17.877370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.879195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.899117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f153ecf-f92a-4b87-bdb4-35d17dce6edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:17.899117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f153ecf-f92a-4b87-bdb4-35d17dce6edb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.899117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f153ecf-f92a-4b87-bdb4-35d17dce6edb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.899117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f153ecf-f92a-4b87-bdb4-35d17dce6edb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.905903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f153ecf-f92a-4b87-bdb4-35d17dce6edb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:17.907316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f153ecf-f92a-4b87-bdb4-35d17dce6edb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:17.908699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 368/749] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:06:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:17.910023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5f153ecf-f92a-4b87-bdb4-35d17dce6edb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:17.928037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:17.929073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:17.930034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 382/750] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:06:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:17.931164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373ca94-86be-4e56-af5b-d55d9bc3952a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:17.932105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373ca94-86be-4e56-af5b-d55d9bc3952a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:17.932105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373ca94-86be-4e56-af5b-d55d9bc3952a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:17.949324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a373ca94-86be-4e56-af5b-d55d9bc3952a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:17.950513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a373ca94-86be-4e56-af5b-d55d9bc3952a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:17.970322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.001593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.025766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 383/751] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:18.026964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373ca94-86be-4e56-af5b-d55d9bc3952a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.043470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff1a66-3f71-4072-968b-33da3401fcb5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.080447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.081282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.081567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.081851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.092356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.093797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:18.113583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:18.114558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:18.115438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:18.115438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:18.115438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:18.115438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:18.115438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 369/752] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:18.115721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.141900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373ca94-86be-4e56-af5b-d55d9bc3952a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.187372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.188172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.188461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.188788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.200654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.202147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.216439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:18.220080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:18.221046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:18.221944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:18.221944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:18.221944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:18.221944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:18.221944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 384/753] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:18.222219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.228801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad4cd0-aba0-4b82-b0e9-af2137fd152f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.261378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.267793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.268760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.269254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.269773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.270737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:18.271054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.272498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.296529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 370/754] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:18.297481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.335004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9871a3-2f56-432b-baeb-c2b2c46c1d1c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.370150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.370887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.371172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.371459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.372197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.373500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:18.386244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:18.387351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:18.388184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 385/755] 10.222.0.52 () {56 vars in 1129 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:18.389857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.413852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac864761-7dff-4227-bad2-8eab7fe04129 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.450730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.451654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.451934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.452221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.452985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:18.453298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.454666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.470210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 371/756] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:18.471025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.489361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.501169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.505375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a5fcf-7003-4b30-8ebd-4a3c47374241 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.545474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.546460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.546745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.547135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.547866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:18.548172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.549833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.563255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 386/757] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:18.564077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.588104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a02587-4fd6-4d52-b81b-f1bca5d2b480 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.625809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.626481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.626829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.627176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.673255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e7e5b-9db7-4ad0-bced-6e63587f0881 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.701869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.703324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.709726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.710590 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.711468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.712476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.713398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:18.714194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.716091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.720875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.721305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 387/758] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:18.722782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.749520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.815219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 372/759] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:06:18.816172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.838364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b642ab-cc01-489c-8198-40075ac543b1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.875806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.876668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.876899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.877359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.884528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.885983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:18.902440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:18.903378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:18.904275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 388/760] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:18.904795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:18.926334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd775a6-4f32-42c1-a5c2-8ba52337918e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:18.940939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:18.964822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:18.965776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:18.966057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:18.966331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:18.973294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:18.974717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:18.989436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:18.990429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:18.991310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 373/761] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:18.992736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.002967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:19.028389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947e576-129a-4592-a443-0aea1bd42fbe None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.066278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.066942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.067224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.067498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.068200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:19.068503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.069886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.075752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 389/762] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:19.080886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.107054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6ac04-f74f-41f0-9472-89fd6650fe76 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.144083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.144825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.145111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.145389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.156705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.158151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:19.175615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:19.176734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:19.177622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:19.177622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:19.177622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:19.177622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:19.177622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 374/763] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:19.177622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.191053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39d0b-56c2-4999-b047-930cc176eb7b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.207711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:19.227712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.228380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.228694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.228972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.229791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:19.230103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.231523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.244550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 390/764] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:19.245311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.258362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:19.294825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ae5a-d7d8-42f2-948c-59f97f32d827 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.328703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.329359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.329670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.329954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.330718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:19.331024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.332440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.349247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 375/765] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:19.349247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.363286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18227f-e9ee-4e75-9f11-ae3d9d681fa8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.401017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.401720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.402005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.402277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.403070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] RBAC: Adding query filter params (interface=public, service_id=a3dcd53d308f43e4ac4c172a74698dbd, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:19.403380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.404989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.418012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 391/766] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/endpoints?service_id=a3dcd53d308f43e4ac4c172a74698dbd&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:19.418012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.434473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:19.450729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae59a9c-09fe-4329-9c58-37b210cb9b85 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.485460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.486116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.486394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.486667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.487378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:19.487687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.489068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.490784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:19.501086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 376/767] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:19.501851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.527838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e60f5-8565-4dd9-b10e-0eafe4b810c0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.561996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.562658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.562943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.563221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.564044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=cbaf3f85739348beb1bc8bae579f0da7) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:19.564355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.565835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.571441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 392/768] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/role_assignments?user.id=cbaf3f85739348beb1bc8bae579f0da7&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:19.571978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.618871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142a2e-1803-4c09-b892-5fc98e8c9632 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.658715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68e69d-19fb-4597-a3dc-99bf56b8e26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.659531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e69d-19fb-4597-a3dc-99bf56b8e26f None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.659812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e69d-19fb-4597-a3dc-99bf56b8e26f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.660089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e69d-19fb-4597-a3dc-99bf56b8e26f None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/48647bb993344a8b91cec7f623aea007/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.662101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68e69d-19fb-4597-a3dc-99bf56b8e26f None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=48647bb993344a8b91cec7f623aea007, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.663773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68e69d-19fb-4597-a3dc-99bf56b8e26f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.682913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 377/769] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:19 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/48647bb993344a8b91cec7f623aea007/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:19.685146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768f3a9-af53-41ff-b72b-43290998ef26 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.685969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768f3a9-af53-41ff-b72b-43290998ef26 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.687002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768f3a9-af53-41ff-b72b-43290998ef26 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.689685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 378/770] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:19.690943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768f3a9-af53-41ff-b72b-43290998ef26 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.701416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8cffb-b38a-4df6-b8a6-6f680ba39a4b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.716973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:19.750543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671532f-29f1-4e33-a8fb-5f90fb62f3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.751905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671532f-29f1-4e33-a8fb-5f90fb62f3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.752338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671532f-29f1-4e33-a8fb-5f90fb62f3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.752847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671532f-29f1-4e33-a8fb-5f90fb62f3e7 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.753759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671532f-29f1-4e33-a8fb-5f90fb62f3e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.755413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671532f-29f1-4e33-a8fb-5f90fb62f3e7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.756948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:19.773087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 393/771] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:19.775139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd1d2-9f13-4f3a-9cb4-479a0bc1ae85 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.775634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd1d2-9f13-4f3a-9cb4-479a0bc1ae85 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.776069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd1d2-9f13-4f3a-9cb4-479a0bc1ae85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.813934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c0fbd1d2-9f13-4f3a-9cb4-479a0bc1ae85 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:19.815440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c0fbd1d2-9f13-4f3a-9cb4-479a0bc1ae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:19.843901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768f3a9-af53-41ff-b72b-43290998ef26 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:19.892526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:19.893364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:19.893642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:19.893911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:19.903711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:19.905537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:19.921752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:19.922679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:19.923912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 379/772] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:19.932605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:19.933879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 394/773] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:19.953666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbd1d2-9f13-4f3a-9cb4-479a0bc1ae85 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:20.004453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:20.042932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58384a29-3982-483a-a6b8-c24756c21acb None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:20.073134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbd1d2-9f13-4f3a-9cb4-479a0bc1ae85 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:20.080829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:20.081608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:20.081962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:20.082308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:20.083291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:20.083468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:20.085048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:20.101081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 380/774] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:20.108352 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:20.109876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:20.121540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:20.122736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:20.123262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:20.123742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] PATH_INFO: `/v3/services/network` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:20.125445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:20.127091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:20.140276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:20.141336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:20.142222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:20.144862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 395/775] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:19 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:20.151411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:20.236995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a16c-ae62-49b2-9d80-7fd56f42a739 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:20.269665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e4425-d78b-49b9-988a-679b78063660 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:20.284974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:20.289397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:20.290664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:20.291125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:20.291567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:20.292615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=b2370d988bdf4499a1a3e71e8336d002) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:20.293151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:20.295579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:20.304716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 381/776] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:20 2022] GET /identity/v3/role_assignments?user.id=b2370d988bdf4499a1a3e71e8336d002&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:20.312953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:20.313810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:20.314193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:20.314572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:20.315407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:20.315872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:20.317453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:20.328843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 396/777] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:20.330301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:20.451114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9be17-0356-44f3-a50c-d4c8c51b2b07 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:20.487595 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:20.488187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:20.488471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:20.488790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:20.489504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] RBAC: Adding query filter params (type=network) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:20.489817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:20.491212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:20.498039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 382/778] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:20.504147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:20.595352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:20.647632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee095a-74db-4ce5-9eb4-462bfe7550d0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:20.696052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cf9cc-6c73-4071-9b1c-32b27b2d3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:20.697677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cf9cc-6c73-4071-9b1c-32b27b2d3def None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:20.698171 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cf9cc-6c73-4071-9b1c-32b27b2d3def None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:20.698671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cf9cc-6c73-4071-9b1c-32b27b2d3def None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:20.699669 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cf9cc-6c73-4071-9b1c-32b27b2d3def None admin] RBAC: Adding query filter params (interface=public, service_id=a8e42843822f43f68e7716ce8334985d, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:20.700185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cf9cc-6c73-4071-9b1c-32b27b2d3def None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:20.701785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cf9cc-6c73-4071-9b1c-32b27b2d3def None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:20.707547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 397/779] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:06:20 2022] GET /identity/v3/endpoints?service_id=a8e42843822f43f68e7716ce8334985d&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:20.712959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:20.788213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:20.860145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedde7d6-f383-4aed-bd8d-c49f74d99a1c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:20.912062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26bb7d-fdef-4313-8283-d3fb0813891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:20.912766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26bb7d-fdef-4313-8283-d3fb0813891e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:20.913054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26bb7d-fdef-4313-8283-d3fb0813891e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:20.913333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26bb7d-fdef-4313-8283-d3fb0813891e None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:20.914013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26bb7d-fdef-4313-8283-d3fb0813891e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:20.915428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26bb7d-fdef-4313-8283-d3fb0813891e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:20.921149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 383/780] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:20 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:21.838034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e4ada-e223-442b-b65b-097961a27378 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:21.838573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e4ada-e223-442b-b65b-097961a27378 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:21.838871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e4ada-e223-442b-b65b-097961a27378 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:21.840666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 398/781] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:21.846937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195827b4-b0e7-4828-bdcc-54c1b8f9936d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:21.847380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195827b4-b0e7-4828-bdcc-54c1b8f9936d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:21.847694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195827b4-b0e7-4828-bdcc-54c1b8f9936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:21.869540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-195827b4-b0e7-4828-bdcc-54c1b8f9936d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:21.870742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-195827b4-b0e7-4828-bdcc-54c1b8f9936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:21.959413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 384/782] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:22.001306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e4ada-e223-442b-b65b-097961a27378 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:22.130415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e4ada-e223-442b-b65b-097961a27378 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:22.151015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:22.167421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:22.168268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.168571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.168859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.180796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:22.188796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:22.189745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:22.191079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 399/783] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:22.197881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195827b4-b0e7-4828-bdcc-54c1b8f9936d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:22.267302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:22.329969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195827b4-b0e7-4828-bdcc-54c1b8f9936d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:22.381351 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa493642-6625-4a05-89e1-83c5268c573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:22.381955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa493642-6625-4a05-89e1-83c5268c573d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.382244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa493642-6625-4a05-89e1-83c5268c573d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.382523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa493642-6625-4a05-89e1-83c5268c573d None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.383274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa493642-6625-4a05-89e1-83c5268c573d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:22.383607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa493642-6625-4a05-89e1-83c5268c573d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:22.385005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa493642-6625-4a05-89e1-83c5268c573d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:22.394284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 385/784] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:22.400525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:22.410285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4e36-b8a2-479d-9362-3a3ea14dd097 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.410572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4e36-b8a2-479d-9362-3a3ea14dd097 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.410852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b4e36-b8a2-479d-9362-3a3ea14dd097 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.414417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 386/785] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:22.419195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9197d87-3e8b-4958-bdd0-2d4131872642 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.419726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9197d87-3e8b-4958-bdd0-2d4131872642 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.420010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9197d87-3e8b-4958-bdd0-2d4131872642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.441743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e9197d87-3e8b-4958-bdd0-2d4131872642 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:22.443005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e9197d87-3e8b-4958-bdd0-2d4131872642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:22.522994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 387/786] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:22.562671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9197d87-3e8b-4958-bdd0-2d4131872642 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:22.575795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27cb78-391c-4cc6-bb33-1dea09d8f529 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:22.620400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc10aa-a8a1-4eeb-8ddb-2f9c938a7eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:22.621312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc10aa-a8a1-4eeb-8ddb-2f9c938a7eed None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.621623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc10aa-a8a1-4eeb-8ddb-2f9c938a7eed None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.621917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc10aa-a8a1-4eeb-8ddb-2f9c938a7eed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.634585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:22.645513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc10aa-a8a1-4eeb-8ddb-2f9c938a7eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:22.646988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc10aa-a8a1-4eeb-8ddb-2f9c938a7eed None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-03bc10aa-a8a1-4eeb-8ddb-2f9c938a7eed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:22.666800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:22.667773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:22.668729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:22.668729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:22.668729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:22.668729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:22.668729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 400/787] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:22.673402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c31a-4438-472e-a399-4c0461244f58 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.673786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c31a-4438-472e-a399-4c0461244f58 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.674155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c31a-4438-472e-a399-4c0461244f58 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.675866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 401/788] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:22.676651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c31a-4438-472e-a399-4c0461244f58 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:22.714412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:22.718761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9197d87-3e8b-4958-bdd0-2d4131872642 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:22.773341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2916b-cb1e-4c2b-90d9-fc5f698d1d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:22.775893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2916b-cb1e-4c2b-90d9-fc5f698d1d96 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.776768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2916b-cb1e-4c2b-90d9-fc5f698d1d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.777254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2916b-cb1e-4c2b-90d9-fc5f698d1d96 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.778321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2916b-cb1e-4c2b-90d9-fc5f698d1d96 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:22.780486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2916b-cb1e-4c2b-90d9-fc5f698d1d96 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:22.797589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c31a-4438-472e-a399-4c0461244f58 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-59c2916b-cb1e-4c2b-90d9-fc5f698d1d96 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:22.799466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:22.800582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:06:22.801423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:22.804808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 388/789] 10.222.0.52 () {56 vars in 1129 bytes} [Fri Jun 24 09:06:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:22.806255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26345dd-da99-449f-b13e-37dd8c825312 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.806688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26345dd-da99-449f-b13e-37dd8c825312 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.807110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26345dd-da99-449f-b13e-37dd8c825312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.828826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e26345dd-da99-449f-b13e-37dd8c825312 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:22.830222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e26345dd-da99-449f-b13e-37dd8c825312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:22.839952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:22.841317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:22.841752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:22.842173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:22.843182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:22.843643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:22.845183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:22.858836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 402/790] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:22.860030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:22.912872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 389/791] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:22.914466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26345dd-da99-449f-b13e-37dd8c825312 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:22.972362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:22.978075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:22.988967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752104-49ec-4bb2-8672-4a99b5b4f505 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.029321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.030424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.030913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.031185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.032001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:23.035400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.037523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.043683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 403/792] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Jun 24 09:06:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:23.044658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.073902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26345dd-da99-449f-b13e-37dd8c825312 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.114540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.115307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.115586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.116005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.123655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.125431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:23.143344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:23.144340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 390/793] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:23.145544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.162275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e592c-984b-4f87-aa57-d7ad80c63994 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.198956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47d70c-5be3-4b7b-82b1-9bb566276f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.199678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d70c-5be3-4b7b-82b1-9bb566276f55 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.199961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d70c-5be3-4b7b-82b1-9bb566276f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.200237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47d70c-5be3-4b7b-82b1-9bb566276f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.210577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47d70c-5be3-4b7b-82b1-9bb566276f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1f47d70c-5be3-4b7b-82b1-9bb566276f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:23.218183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:23.224743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 404/794] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:23.225573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c126-cc73-44bf-8cd8-6037186da351 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.226444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c126-cc73-44bf-8cd8-6037186da351 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.226444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c126-cc73-44bf-8cd8-6037186da351 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.226444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 405/795] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:23.226444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c126-cc73-44bf-8cd8-6037186da351 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.245037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:23.259190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:23.267626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db196111-e99e-4013-8c44-c818d1f09d69 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.314104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.314788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.315073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.315336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.316046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:23.316356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.317773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.323142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 391/796] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:23.324137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.350860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c126-cc73-44bf-8cd8-6037186da351 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.389008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b9c22-1366-4134-a6b4-e9afc50d9bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.390168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b9c22-1366-4134-a6b4-e9afc50d9bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.390633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b9c22-1366-4134-a6b4-e9afc50d9bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.391188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b9c22-1366-4134-a6b4-e9afc50d9bd9 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.392333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b9c22-1366-4134-a6b4-e9afc50d9bd9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:23.392840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b9c22-1366-4134-a6b4-e9afc50d9bd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.394444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b9c22-1366-4134-a6b4-e9afc50d9bd9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.413532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 406/797] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:23.415588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9841be-d61e-435a-889d-5b5875c8bda1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.415873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9841be-d61e-435a-889d-5b5875c8bda1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.416145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9841be-d61e-435a-889d-5b5875c8bda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.434036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5bcb0-7acb-415c-ac4c-1dcb9b85cbcd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.434542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-aa9841be-d61e-435a-889d-5b5875c8bda1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:23.435734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-aa9841be-d61e-435a-889d-5b5875c8bda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:23.472484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.473229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.473572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.473913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.474812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:23.475178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.476599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.485971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 392/798] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:23.486760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.511213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:23.516888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 407/799] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:23.517773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9841be-d61e-435a-889d-5b5875c8bda1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.521751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:23.601874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0b184-907d-4da0-ad0d-495de540d33d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.622104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9841be-d61e-435a-889d-5b5875c8bda1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.636926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.637671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.637960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.638241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.638985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.640348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.661073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bdd86-c689-43fb-88e5-65d32651e1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.671484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 393/800] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:06:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:23.671713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bdd86-c689-43fb-88e5-65d32651e1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.671713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bdd86-c689-43fb-88e5-65d32651e1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.671713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bdd86-c689-43fb-88e5-65d32651e1c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.671954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.680117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bdd86-c689-43fb-88e5-65d32651e1c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.681682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bdd86-c689-43fb-88e5-65d32651e1c7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-442bdd86-c689-43fb-88e5-65d32651e1c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:23.703828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:23.705248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:23.706467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:23.706467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:23.706467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:23.706467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:23.706467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 408/801] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:23.707208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1264-b174-4ce1-9a2c-d2beff2008ea None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.707208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1264-b174-4ce1-9a2c-d2beff2008ea None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.707434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1264-b174-4ce1-9a2c-d2beff2008ea None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.709038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 409/802] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:23.709832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1264-b174-4ce1-9a2c-d2beff2008ea None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.774073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:23.777446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:23.813919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3124-8254-4831-b82c-2caf2cafd3e0 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.833364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1264-b174-4ce1-9a2c-d2beff2008ea None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:23.857305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.858112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.858398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.858776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.872472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:23.874017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:23.880076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8bac5-7bb1-4330-9224-518f6a9a7d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:23.880908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8bac5-7bb1-4330-9224-518f6a9a7d18 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.881189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8bac5-7bb1-4330-9224-518f6a9a7d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.881461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8bac5-7bb1-4330-9224-518f6a9a7d18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:23.891436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:23.892492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:23.893463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:23.893463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:23.893463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:23.893463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:23.893463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 394/803] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:23.893463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:23.894926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8bac5-7bb1-4330-9224-518f6a9a7d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-22d8bac5-7bb1-4330-9224-518f6a9a7d18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:23.903469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:23.904599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:23.905505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 410/804] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:23.906306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2388b8-874c-45f0-b3a0-e5ac2dba5e25 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:23.906594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2388b8-874c-45f0-b3a0-e5ac2dba5e25 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:23.906871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2388b8-874c-45f0-b3a0-e5ac2dba5e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:23.924145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9c2388b8-874c-45f0-b3a0-e5ac2dba5e25 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:23.925368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9c2388b8-874c-45f0-b3a0-e5ac2dba5e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:23.997152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 411/805] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:23.998509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2388b8-874c-45f0-b3a0-e5ac2dba5e25 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.027599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ca8e3-129f-4e79-8e48-5cda29c3ed94 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.061148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.069504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.077490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.078162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.078442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.078713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.079548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:24.079971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.081354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.094456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 395/806] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:24.095254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.129071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2388b8-874c-45f0-b3a0-e5ac2dba5e25 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.171097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.171768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.172054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.172332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.173096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:24.173401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.174782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.186798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 412/807] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:24.189816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.225402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96552ca5-55c9-4c84-8aea-dcf01f22a3e3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.265160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.265829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.266110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.266382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.267126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:24.267431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.268808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.282581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 396/808] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:24.282581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.308340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169faab4-b6be-4c49-84db-5b9714907b9d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.316589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.330469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.350522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.351651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.352316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.352625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.353443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.354919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:24.367600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:24.369203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:24.375391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 413/809] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:24.376196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.404788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d92cff-7edf-416b-aa4a-d53720650343 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.441883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.443032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.443152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.443467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.451153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.452730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:24.467279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:24.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:24.469361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 397/810] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:24.469832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.494829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9022663-feeb-43f9-b682-64a76e3c3ef3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.535517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.535937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.536216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.536487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.544199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.545717 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:24.561081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:24.562206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 414/811] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:24.568041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.586034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bd162-6bf7-4fc3-9f19-60d22b921bb5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.596384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.598860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.629136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.630310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.630893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.631483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.643288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.645312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:24.666275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:24.667198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:24.668138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:24.668138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:24.668138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:24.668138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:24.669972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 398/812] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:24.670904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.685658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d70b10-9232-4ff7-a509-5ace5433a75e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.726391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.727089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.727400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.728181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.728964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] RBAC: Adding query filter params (name=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:24.729225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.730612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.735595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 415/813] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:24.740639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.787363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f109c-b539-4d04-9b3c-d9e70d04719e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.828920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.829729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.830055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.830338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.831126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:24.831430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.832903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.846843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 399/814] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:24.847733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.867626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.871372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:24.877960 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ac958-a988-4401-8dec-35ad98844b0c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:24.915177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:24.916599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:24.916923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:24.917202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:24.918032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:24.918337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:24.919854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:24.943138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 416/815] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:24.943138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:24.969582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d94dbe-375a-40ec-97fe-987a7a991c6d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.012901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.013574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.013857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.014135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.014889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:25.015200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.016650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.047315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 400/816] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:25.047315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:25.062641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a0741-7342-4ae8-a5d5-e1bbeaffd129 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.099323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.099986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.100265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.100538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.101276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] RBAC: Adding query filter params (type=network) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:25.101582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.102964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.112098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 417/817] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:25.112098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:25.129272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:25.140918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:25.165692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6092f4f-4fa7-4ac2-bfd4-963ba18b7059 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.203333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.204331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.204716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.205070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/cbaf3f85739348beb1bc8bae579f0da7/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.207145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=cbaf3f85739348beb1bc8bae579f0da7, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.208712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.223554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 401/818] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:25 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/cbaf3f85739348beb1bc8bae579f0da7/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:25.224292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:25.251690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d213-8633-4ce7-a6be-565561ffe88a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.291958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430962b-fe56-4f82-9407-970392244503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.292860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430962b-fe56-4f82-9407-970392244503 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.293287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430962b-fe56-4f82-9407-970392244503 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.293606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430962b-fe56-4f82-9407-970392244503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.380230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b128aa3-3b5a-480a-865f-fa3ca90a5907 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.381618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430962b-fe56-4f82-9407-970392244503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.383117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430962b-fe56-4f82-9407-970392244503 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.406773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:25.426113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.426945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.427230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.427508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.438497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.439961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.454410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:25.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:25.459769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:25.460970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:25.461585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 402/819] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:06:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:25.462328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:25.530150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 418/820] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:06:25.531893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430962b-fe56-4f82-9407-970392244503 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:25.577645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a63059-7f78-4122-8e81-4b02e237e330 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.626543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.627324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.627683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.628036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.628890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.630447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.657033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 403/821] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:06:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:25.658013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:25.670566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430962b-fe56-4f82-9407-970392244503 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.703744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:25.734757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:25.755324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.756122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.756590 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.756940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.757855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:25.758288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.760369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.796717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f4e57-1d18-45ac-8024-aa23795097bd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.798037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 419/822] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Jun 24 09:06:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:25.803836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:25.845240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cd050-716f-4b95-8020-b945ddb22050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.846063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd050-716f-4b95-8020-b945ddb22050 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.846354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd050-716f-4b95-8020-b945ddb22050 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.846748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd050-716f-4b95-8020-b945ddb22050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.940144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86774f7b-a026-4340-ac3a-de732c5b929c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:25.978504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:25.979328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:25.979621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:25.979905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:25.985245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cd050-716f-4b95-8020-b945ddb22050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.986853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cd050-716f-4b95-8020-b945ddb22050 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:25.994073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:25.996190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-329cd050-716f-4b95-8020-b945ddb22050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:26.010930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:26.012170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:26.013204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:26.013204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:26.013204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:26.013204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:26.013204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 404/823] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:26.014148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0276f64-e195-41a5-a485-d95fda22053b None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.014287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0276f64-e195-41a5-a485-d95fda22053b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.014565 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0276f64-e195-41a5-a485-d95fda22053b None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.016603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 405/824] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:26.022587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa33a-b493-4a78-8681-5213b6e0710d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.023007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa33a-b493-4a78-8681-5213b6e0710d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.023392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa33a-b493-4a78-8681-5213b6e0710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:26.024790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:26.025767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:26.026774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:26.026774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:26.026774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:26.026774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:26.026774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 420/825] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:26.027553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.052238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-700fa33a-b493-4a78-8681-5213b6e0710d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:26.053478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-700fa33a-b493-4a78-8681-5213b6e0710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:26.059377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:26.127023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 406/826] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:26.128004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fa33a-b493-4a78-8681-5213b6e0710d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.154796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a39d-9ed7-4e62-99de-f0221bedd93b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.165610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:26.195246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.196138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.196610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.197043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.197904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:26.198374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.199926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.213008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 421/827] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:26.213815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.254456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fa33a-b493-4a78-8681-5213b6e0710d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.290109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.291217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.291217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.291492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.292207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:26.292510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.293889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.304481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:26.311921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 407/828] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:26.311921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.331100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021b0c0-8237-493f-a94e-f4086b9c1d49 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.372341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.373183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.373526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.373797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.374473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.375826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:26.388501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:26.389585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:26.390563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 422/829] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:26.391463 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.424203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:26.431269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31e51f-9c15-440e-8ab0-f40723f249a4 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.468917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.469888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.470523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.470802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.477898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.479412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:26.494900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:26.496077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:26.497197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 408/830] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:26.497700 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.527679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b5067-7f1b-4183-b054-c65088f995f9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.566630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.567525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.567722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.567989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.576190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.577682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:26.595561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:26.596488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:26.597465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 423/831] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:26.597950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.607423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:26.636753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c7759-984d-44d9-81a2-c2e66ef0fde8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.673711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.674290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.674570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.674846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.675541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:26.675840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.677221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.687524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 409/832] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:26.687524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.713013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:26.724419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59e814-93c8-4d3a-af35-9b6bde8bf2e5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.762354 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.763021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.763307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.763584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.764370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:26.764725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.766160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.781220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 424/833] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:26.782211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.812426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421afc82-e6f0-4a12-92bc-613715eaad67 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.852924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.853593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.853875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.854341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.854994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:26.855238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.856691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.861421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:26.875085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 410/834] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:26.875085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.909299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d222799-629b-4d14-b044-d7b3effed615 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:26.947606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:26.948713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:26.949010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:26.949292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:26.950303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:26.950616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:26.953033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:26.958801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 425/835] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:26.959658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:26.982594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:27.010314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fbd45-26ef-4ed5-8b1a-bdcdf319711d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:27.074851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b1c71-2396-4456-92f0-f757b700be94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:27.077052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1c71-2396-4456-92f0-f757b700be94 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:27.077338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1c71-2396-4456-92f0-f757b700be94 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:27.077613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1c71-2396-4456-92f0-f757b700be94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:27.078473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b1c71-2396-4456-92f0-f757b700be94 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=48647bb993344a8b91cec7f623aea007) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:27.078785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b1c71-2396-4456-92f0-f757b700be94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:27.080288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b1c71-2396-4456-92f0-f757b700be94 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:27.097027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 411/836] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:26 2022] GET /identity/v3/role_assignments?user.id=48647bb993344a8b91cec7f623aea007&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:27.119193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833ac68-6b47-4776-8de0-f11199bdda17 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:27.155519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:27.161664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033f1cd-72e6-4aea-95da-9ff5468ac0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:27.162702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f1cd-72e6-4aea-95da-9ff5468ac0ae None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:27.163033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f1cd-72e6-4aea-95da-9ff5468ac0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:27.163362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033f1cd-72e6-4aea-95da-9ff5468ac0ae None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/b2370d988bdf4499a1a3e71e8336d002/roles/9654f3986243401db4f1aad536273ede` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:27.185064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033f1cd-72e6-4aea-95da-9ff5468ac0ae None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=b2370d988bdf4499a1a3e71e8336d002, role_id=9654f3986243401db4f1aad536273ede)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:27.186759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033f1cd-72e6-4aea-95da-9ff5468ac0ae None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:27.204065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 426/837] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:26 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/b2370d988bdf4499a1a3e71e8336d002/roles/9654f3986243401db4f1aad536273ede => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:27.501740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a2aa0-68f6-41ed-a47c-8c6e8b3377cb None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:27.501934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a2aa0-68f6-41ed-a47c-8c6e8b3377cb None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:27.502204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a2aa0-68f6-41ed-a47c-8c6e8b3377cb None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:27.503820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 412/838] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:27.509412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93ba02-e92a-47b3-9eb2-fd223abbcde4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:27.509701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93ba02-e92a-47b3-9eb2-fd223abbcde4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:27.509979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93ba02-e92a-47b3-9eb2-fd223abbcde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:27.527750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7d93ba02-e92a-47b3-9eb2-fd223abbcde4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:27.528979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7d93ba02-e92a-47b3-9eb2-fd223abbcde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:27.606369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 427/839] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:27.629863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a2aa0-68f6-41ed-a47c-8c6e8b3377cb None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:27.748588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a2aa0-68f6-41ed-a47c-8c6e8b3377cb None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:27.785509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:27.786314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:27.786600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:27.786874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:27.797907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:27.805496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:27.806449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:27.807300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 413/840] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:27.813642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93ba02-e92a-47b3-9eb2-fd223abbcde4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:27.919521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93ba02-e92a-47b3-9eb2-fd223abbcde4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:27.932322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:27.958081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:27.958889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:27.959041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:27.959349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:27.960209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:27.960517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:27.962542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:27.974205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 428/841] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:27.979949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:27.984137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:28.104821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35174105-9960-46fb-aeeb-3b59b0318101 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:28.142564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:28.143405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:28.143690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:28.143965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:28.154871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:28.156320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:28.174421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:28.175388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:28.176360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:28.176360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:28.176360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:28.176360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:28.176360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 414/842] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:28.182288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:28.285303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23384e-2dc6-4147-9fd6-3ab584812b4a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:28.320288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:28.320940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:28.321821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:28.322183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:28.322986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:28.323289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:28.324665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:28.337629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 429/843] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:28.343915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:28.409851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:28.456395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca309303-2ca7-4ada-92a2-86ec6835403b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:28.490994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:28.492121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:28.492607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:28.493615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:28.501920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:28.503560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:28.518064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:28.519014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:06:28.519931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:28.522204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 415/844] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:28.527938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:28.566481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:28.635285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56537e24-45cf-4662-a15f-bc9d7a9fc76b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:28.670064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:28.670742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:28.671169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:28.671455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:28.672287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:28.672616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:28.674029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:28.693462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 430/845] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Jun 24 09:06:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:28.702197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:28.807529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5862aa-853d-4681-9f84-dc5ee23d30da None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:28.824888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:28.844939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:28.845599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:28.845881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:28.846155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:28.920122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:28.921648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:29.032333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 416/846] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:06:29.037874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.138155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:29.150277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da92dd-da5c-4955-a5c4-47243d6f2b12 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:29.189095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7d0e3-7148-43e6-8355-18a2e02fbfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:29.190226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7d0e3-7148-43e6-8355-18a2e02fbfeb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.190725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7d0e3-7148-43e6-8355-18a2e02fbfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.191193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7d0e3-7148-43e6-8355-18a2e02fbfeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.202530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7d0e3-7148-43e6-8355-18a2e02fbfeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:29.204304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7d0e3-7148-43e6-8355-18a2e02fbfeb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-dcc7d0e3-7148-43e6-8355-18a2e02fbfeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:29.223606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:29.224609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:29.225514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:29.225514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:29.225514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:29.225514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:29.228386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 431/847] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:29.236349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.238018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2cbc3-77c5-40f3-a19c-1786fd6eb478 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.238304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2cbc3-77c5-40f3-a19c-1786fd6eb478 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.238585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2cbc3-77c5-40f3-a19c-1786fd6eb478 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.240058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 432/848] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:29.245798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3f3c-5d7b-4220-8e2a-9e37993d03b7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.246368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3f3c-5d7b-4220-8e2a-9e37993d03b7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.246819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3f3c-5d7b-4220-8e2a-9e37993d03b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.258262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:29.264858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cddb3f3c-5d7b-4220-8e2a-9e37993d03b7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:29.266264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cddb3f3c-5d7b-4220-8e2a-9e37993d03b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:29.350456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 433/849] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:29.356629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbafc6c-f392-4745-98d2-389b832a9157 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:29.358121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d349c-687c-4b83-9da3-f14368dc46fb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.358422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d349c-687c-4b83-9da3-f14368dc46fb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.358708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d349c-687c-4b83-9da3-f14368dc46fb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.360853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 434/850] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:29.365369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9dffd-3a90-4d52-be4d-174009b38892 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.365661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9dffd-3a90-4d52-be4d-174009b38892 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.365946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9dffd-3a90-4d52-be4d-174009b38892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.383049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-02b9dffd-3a90-4d52-be4d-174009b38892 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:29.384860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-02b9dffd-3a90-4d52-be4d-174009b38892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:29.400941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:29.401973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.402555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.403298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.404201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:29.404705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:29.406710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:29.421892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 417/851] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:29.422916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.455995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 435/852] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:29.456873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9dffd-3a90-4d52-be4d-174009b38892 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.474781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:29.512710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:29.539513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ab797-1592-4a90-884c-1d8b618a5cae None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:29.569521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9dffd-3a90-4d52-be4d-174009b38892 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:29.573335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:29.574012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.574299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.574577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.575297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:29.576688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:29.590404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 418/853] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:29 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:29.591209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.615750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961de678-0947-4dbb-8655-b3cfc8aa9330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:29.616653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961de678-0947-4dbb-8655-b3cfc8aa9330 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.616902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961de678-0947-4dbb-8655-b3cfc8aa9330 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.617183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961de678-0947-4dbb-8655-b3cfc8aa9330 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.627060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961de678-0947-4dbb-8655-b3cfc8aa9330 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:29.629157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961de678-0947-4dbb-8655-b3cfc8aa9330 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-961de678-0947-4dbb-8655-b3cfc8aa9330 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:29.644684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:29.645724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:29.646735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 436/854] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:29.647623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df0e12-34bf-47aa-805f-00a5dc4df95b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.647925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df0e12-34bf-47aa-805f-00a5dc4df95b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.648144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df0e12-34bf-47aa-805f-00a5dc4df95b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.652601 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 437/855] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:29.653928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df0e12-34bf-47aa-805f-00a5dc4df95b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.717845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bf3f5-bfee-4d43-9cd1-f4f01f0bcace None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:29.726237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:29.763772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22869885-b64c-45ed-81cf-7958cc3f09d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:29.764211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869885-b64c-45ed-81cf-7958cc3f09d4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.764648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869885-b64c-45ed-81cf-7958cc3f09d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.765073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869885-b64c-45ed-81cf-7958cc3f09d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.768723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:29.782183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22869885-b64c-45ed-81cf-7958cc3f09d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:29.786195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df0e12-34bf-47aa-805f-00a5dc4df95b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-22869885-b64c-45ed-81cf-7958cc3f09d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:29.789447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:29.792827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 419/856] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:06:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:29.795985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac779f-5120-4df6-8a44-5f2736df9011 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.798577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac779f-5120-4df6-8a44-5f2736df9011 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.798577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac779f-5120-4df6-8a44-5f2736df9011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.812711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-51ac779f-5120-4df6-8a44-5f2736df9011 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:29.813998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-51ac779f-5120-4df6-8a44-5f2736df9011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:29.835331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:29.836217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:29.836343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:29.836604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:29.837390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:29.837697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:29.839134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:29.856670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 438/857] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:29.856670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.892615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 420/858] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:29 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:29.893702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac779f-5120-4df6-8a44-5f2736df9011 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:29.971800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ebe44-f68e-47b4-a82b-3be34a604c25 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.013726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac779f-5120-4df6-8a44-5f2736df9011 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.054389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:30.067088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.067780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.068074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.068356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.069572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=cbaf3f85739348beb1bc8bae579f0da7) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:30.070040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.071639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.088944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 421/859] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:29 2022] GET /identity/v3/role_assignments?user.id=cbaf3f85739348beb1bc8bae579f0da7&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:30.088944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:30.117006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:30.140748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.141586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.141872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.142151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.142935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:30.143301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.145597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.158511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 439/860] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:06:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:30.163840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:30.215578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6ed5-f58f-4c65-b67c-b70f6a325d54 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.254156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.255691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.256185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.256660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.257493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.259066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:30.273361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:30.274395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:30.275246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 422/861] 10.222.0.52 () {56 vars in 1132 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:30.281030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:30.283347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98136cc1-ea2c-4c83-b235-8e52de7f373c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.333247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.335199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.335698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.336200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.348859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:30.357317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.359842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:30.383593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:30.384584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:30.385485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:30.385485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:30.385485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:30.385485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:30.390658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 440/862] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:30.392963 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:30.411327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893e086-0a4b-4b16-9c55-a1aa23f21003 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.426346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:30.452545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.453809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.454313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.454793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.455705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:30.456264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.458017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.464403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 423/863] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:30.469853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:30.504872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cc064-5e0b-43a5-a91b-3616f0cec938 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.540547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.541263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.541545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.542090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.542811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:30.543113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.544682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.560660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 441/864] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:30.566000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:30.594070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519b42a-bcd3-45ce-afa9-610509161cac None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.615998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:30.635893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.636590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.636896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.637183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.637895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:30.638207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.639625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.645222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 424/865] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:30.687710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:30.698592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7fa58-2d71-4c3e-a28e-162a0a30b61e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.742973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.743818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.744101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.744377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.752903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.754369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:30.771756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:30.772756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:30.774871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 442/866] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:30.779490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:30.901506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07bdaa-3478-48f7-abf3-845f623d6149 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:30.941418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:30.942146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:30.942432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:30.942708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:30.943546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:30.943851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:30.945307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:30.965189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 425/867] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:30.970679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:31.081281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99302c29-9688-4166-bfcd-b91fccb9cefb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:31.122565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:31.123324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:31.123639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:31.123793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:31.148182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:31.198831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:31.200293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:31.307140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 443/868] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:30 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:06:31.312864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:31.361582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:31.427254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea13dce-bbdc-4a36-a5b4-1f31dfa6e3f2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:31.466346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:31.467154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:31.467443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:31.467788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:31.479761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:31.481369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:31.500252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:31.502463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:31.504681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:31.504681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:31.504681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:31.504681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:31.504681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 426/869] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:31.509788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:31.571011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:31.621223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c8ae8-df89-4fbf-b31e-cd93659ff7e0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:31.657599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ac0ec-9517-4b55-8c02-54970fde6bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:31.658649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ac0ec-9517-4b55-8c02-54970fde6bcf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:31.659139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ac0ec-9517-4b55-8c02-54970fde6bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:31.659621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ac0ec-9517-4b55-8c02-54970fde6bcf None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:31.660588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ac0ec-9517-4b55-8c02-54970fde6bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:31.661122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ac0ec-9517-4b55-8c02-54970fde6bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:31.662705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ac0ec-9517-4b55-8c02-54970fde6bcf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:31.678297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 444/870] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:31.684266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:31.781016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a40e-98fb-4da1-a520-75bac8039a47 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:31.781477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a40e-98fb-4da1-a520-75bac8039a47 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:31.782153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a40e-98fb-4da1-a520-75bac8039a47 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:31.784061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 445/871] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:31.784233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:31.790168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17980d7b-93d8-4337-abe4-a107ca6e7a54 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:31.790596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17980d7b-93d8-4337-abe4-a107ca6e7a54 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:31.791045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17980d7b-93d8-4337-abe4-a107ca6e7a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:31.794736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76088c-b511-4dc9-b2e7-145b714a8b34 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:31.809812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-17980d7b-93d8-4337-abe4-a107ca6e7a54 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:31.811100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-17980d7b-93d8-4337-abe4-a107ca6e7a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:31.832085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:31.832843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:31.833120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:31.833393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:31.841170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:31.842621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:31.860374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:31.861414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:31.862313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:31.862931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 427/872] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:31.868348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:31.892613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 446/873] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:31.917541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17980d7b-93d8-4337-abe4-a107ca6e7a54 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:31.969023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:32.007015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f1bb1-5448-471b-a6f2-45c41c0e2144 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.029652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17980d7b-93d8-4337-abe4-a107ca6e7a54 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.044658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.045653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.046229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.046693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.047696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:32.048192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.050512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.068868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 428/874] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:32.071966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.072978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.073784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.074070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.074341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.075001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.076722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.089570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:32.093433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:32.094445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:32.095291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:32.100005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 447/875] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:32.101321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.199498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089cf5b-2991-41f7-9478-0c9d621d4dd2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.236688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0655c-4a7a-4236-8721-8c4ba6808d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.237753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0655c-4a7a-4236-8721-8c4ba6808d87 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.238258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0655c-4a7a-4236-8721-8c4ba6808d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.238739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0655c-4a7a-4236-8721-8c4ba6808d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.239788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0655c-4a7a-4236-8721-8c4ba6808d87 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=b2370d988bdf4499a1a3e71e8336d002) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:32.240309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0655c-4a7a-4236-8721-8c4ba6808d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.241992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0655c-4a7a-4236-8721-8c4ba6808d87 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.244791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df4729-6c4f-4336-b475-1e010ecd46fc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.254756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:32.255400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 429/876] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/role_assignments?user.id=b2370d988bdf4499a1a3e71e8336d002&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:32.287836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.288530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.288847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.289124 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.290505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:32.290817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.291949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973d298-18b8-411d-8ea4-7d2a7abedfd3 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.292244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973d298-18b8-411d-8ea4-7d2a7abedfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.292347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.292528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973d298-18b8-411d-8ea4-7d2a7abedfd3 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.294145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 430/877] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:32.298022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 448/878] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:32.300657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2e660-7474-43d4-b641-74ad3d466330 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.301185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2e660-7474-43d4-b641-74ad3d466330 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.301658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2e660-7474-43d4-b641-74ad3d466330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.303591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.322112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-53e2e660-7474-43d4-b641-74ad3d466330 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:32.324215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-53e2e660-7474-43d4-b641-74ad3d466330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:32.412672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 431/879] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:32.431386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2e660-7474-43d4-b641-74ad3d466330 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.439980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:32.442689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925a6a2-3b13-48c9-a3ab-f98a68ffedfb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.491434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.493457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.494240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.494729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.495855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:32.496382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.498572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.506789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 449/880] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:32.513310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.556995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:32.563491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2e660-7474-43d4-b641-74ad3d466330 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.613740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.614560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.615203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.615508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.616177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.618193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:32.634474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:32.635538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:32.636379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 432/881] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:32.643995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d854bb-6631-48c7-89e5-e0a3699eb4e1 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.644912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.689629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.690905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.692339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.692872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.693873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] RBAC: Adding query filter params (interface=public, service_id=b8fa6a1478254841844750272edb2bfe, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:32.694381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.696023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.703872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 450/882] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/endpoints?service_id=b8fa6a1478254841844750272edb2bfe&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:32.712641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.737891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:32.771848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818047-6fbe-43d3-abdf-261309ba4a22 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.810443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.811522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.812083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.812547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.813458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:32.813946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.815519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.816061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:32.821422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 433/883] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:32.821771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d381da-dc17-46d7-8153-824a54d8df47 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:32.830852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:32.861962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae65ad-4fe0-4b18-b89e-683d7442876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:32.862646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae65ad-4fe0-4b18-b89e-683d7442876f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.862933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae65ad-4fe0-4b18-b89e-683d7442876f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.863214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae65ad-4fe0-4b18-b89e-683d7442876f None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.863905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae65ad-4fe0-4b18-b89e-683d7442876f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:32.865545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae65ad-4fe0-4b18-b89e-683d7442876f None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:32.872133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 451/884] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/services => generated 1011 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:06:32.882176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293b43-43cd-4fdb-8437-c2a411256142 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.882519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293b43-43cd-4fdb-8437-c2a411256142 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.882738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293b43-43cd-4fdb-8437-c2a411256142 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.884579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 452/885] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:32.892588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad6620-ce76-4c35-af1a-4cc3eb413cda None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:32.892588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad6620-ce76-4c35-af1a-4cc3eb413cda None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:32.892588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad6620-ce76-4c35-af1a-4cc3eb413cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:32.911301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5dad6620-ce76-4c35-af1a-4cc3eb413cda None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:32.912740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5dad6620-ce76-4c35-af1a-4cc3eb413cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:32.965606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b62b68-e98b-4d2c-b76b-437c08aed5a8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:33.007743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f72810-e26d-49f5-b0e9-6420337ac2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:33.009035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72810-e26d-49f5-b0e9-6420337ac2ef None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:33.009325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72810-e26d-49f5-b0e9-6420337ac2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:33.009605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f72810-e26d-49f5-b0e9-6420337ac2ef None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:33.010336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72810-e26d-49f5-b0e9-6420337ac2ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:33.010693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 453/886] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:33.017114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72810-e26d-49f5-b0e9-6420337ac2ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:33.018603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f72810-e26d-49f5-b0e9-6420337ac2ef None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:33.022198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:33.028681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 434/887] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:33.037430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad6620-ce76-4c35-af1a-4cc3eb413cda None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:33.153425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:33.169575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad6620-ce76-4c35-af1a-4cc3eb413cda None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:33.209258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f5a0d-e683-4107-b9a2-f5592238f515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:33.210217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f5a0d-e683-4107-b9a2-f5592238f515 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:33.210653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f5a0d-e683-4107-b9a2-f5592238f515 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:33.211082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f5a0d-e683-4107-b9a2-f5592238f515 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:33.212576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f5a0d-e683-4107-b9a2-f5592238f515 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:33.214120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f5a0d-e683-4107-b9a2-f5592238f515 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:33.231110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 454/888] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:33 2022] POST /identity/v3/services => generated 269 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:06:34.532536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fc4a1-8d8e-4cd7-9f8a-fe6dc8de1447 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:34.533072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fc4a1-8d8e-4cd7-9f8a-fe6dc8de1447 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:34.533153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fc4a1-8d8e-4cd7-9f8a-fe6dc8de1447 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:34.534757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 435/889] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:34.540214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eda6d-7cfa-4c0a-af18-1d071386d4c2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:34.541163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eda6d-7cfa-4c0a-af18-1d071386d4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:34.541667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eda6d-7cfa-4c0a-af18-1d071386d4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:34.561152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0d4eda6d-7cfa-4c0a-af18-1d071386d4c2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:34.562989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0d4eda6d-7cfa-4c0a-af18-1d071386d4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:34.645720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 455/890] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:34.671223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fc4a1-8d8e-4cd7-9f8a-fe6dc8de1447 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:34.790668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fc4a1-8d8e-4cd7-9f8a-fe6dc8de1447 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:34.833228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:34.834055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:34.834439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:34.834726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:34.847468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:34.856069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:34.857128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:34.858036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 436/891] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Jun 24 09:06:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:34.865301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eda6d-7cfa-4c0a-af18-1d071386d4c2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:34.952278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:34.979701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eda6d-7cfa-4c0a-af18-1d071386d4c2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:35.021251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d9dbf-db00-4a6a-91e9-8f7d20653997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:35.022319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9dbf-db00-4a6a-91e9-8f7d20653997 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.022832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9dbf-db00-4a6a-91e9-8f7d20653997 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.023314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9dbf-db00-4a6a-91e9-8f7d20653997 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.024510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9dbf-db00-4a6a-91e9-8f7d20653997 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:35.025060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9dbf-db00-4a6a-91e9-8f7d20653997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:35.026680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9dbf-db00-4a6a-91e9-8f7d20653997 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:35.039322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 456/892] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Jun 24 09:06:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:35.045247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.100426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b87fa-4ab9-43ab-9c7b-eb716ed413d7 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.101118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b87fa-4ab9-43ab-9c7b-eb716ed413d7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.101609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b87fa-4ab9-43ab-9c7b-eb716ed413d7 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.103571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 457/893] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:35.108910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c938c6-5f59-4819-b926-515990f8c69e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.109197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c938c6-5f59-4819-b926-515990f8c69e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.109769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c938c6-5f59-4819-b926-515990f8c69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.113233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:35.130353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d6c938c6-5f59-4819-b926-515990f8c69e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:35.131540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d6c938c6-5f59-4819-b926-515990f8c69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:35.159353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b638d1-9cf3-4d44-83fd-f24be58252fd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:35.197567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:35.198479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.198763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.199040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.203010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 458/894] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:35.210171 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc1fc7-aa2f-42a5-b19e-6a0cf18decbb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.210348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc1fc7-aa2f-42a5-b19e-6a0cf18decbb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.210966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc1fc7-aa2f-42a5-b19e-6a0cf18decbb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.212425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 459/895] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:35.216530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:35.217726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0984e-ebe4-467c-a390-bf6b34dfba6a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.218018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0984e-ebe4-467c-a390-bf6b34dfba6a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.218295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0984e-ebe4-467c-a390-bf6b34dfba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.218983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:35.236519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fcf0984e-ebe4-467c-a390-bf6b34dfba6a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:35.237741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fcf0984e-ebe4-467c-a390-bf6b34dfba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:35.239751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:35.240804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:35.241709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:35.241709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:35.241709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:35.241709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:35.244877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 437/896] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:35.246011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.314591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:35.328907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 460/897] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:35.331385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0984e-ebe4-467c-a390-bf6b34dfba6a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.363151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856950ad-e1a2-4536-ba23-de4a7b4e3155 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:35.374052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:35.399647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:35.400610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.400895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.401327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.402441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:35.403809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:35.419077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 438/898] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:35 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:35.429645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.450963 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0984e-ebe4-467c-a390-bf6b34dfba6a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:35.500255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15e0ca-04e0-4229-bb58-c4938f632227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:35.500968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15e0ca-04e0-4229-bb58-c4938f632227 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.501252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15e0ca-04e0-4229-bb58-c4938f632227 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.501528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15e0ca-04e0-4229-bb58-c4938f632227 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.502270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15e0ca-04e0-4229-bb58-c4938f632227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:35.502578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15e0ca-04e0-4229-bb58-c4938f632227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:35.503996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15e0ca-04e0-4229-bb58-c4938f632227 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:35.521431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 461/899] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:35.521431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23bdbb-ae3d-45f0-80e3-bea38f5f9ccb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.521431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23bdbb-ae3d-45f0-80e3-bea38f5f9ccb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.521431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23bdbb-ae3d-45f0-80e3-bea38f5f9ccb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.521431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 462/900] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:35.525558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23bdbb-ae3d-45f0-80e3-bea38f5f9ccb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.582876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b849d-780b-4efa-b8c4-9afd6605ed49 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:35.598473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:35.626365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8060d8-a034-4a69-acd9-a68d61135215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:35.627187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8060d8-a034-4a69-acd9-a68d61135215 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.627797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8060d8-a034-4a69-acd9-a68d61135215 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.628079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8060d8-a034-4a69-acd9-a68d61135215 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.629078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8060d8-a034-4a69-acd9-a68d61135215 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:35.631202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8060d8-a034-4a69-acd9-a68d61135215 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-3f8060d8-a034-4a69-acd9-a68d61135215 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:35.647452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:35.650681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:35.652217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 439/901] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:35.654628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bace41f-402e-41be-90b9-64991aff4784 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.654628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bace41f-402e-41be-90b9-64991aff4784 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.654628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bace41f-402e-41be-90b9-64991aff4784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.656091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23bdbb-ae3d-45f0-80e3-bea38f5f9ccb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:35.674143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-4bace41f-402e-41be-90b9-64991aff4784 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:35.675330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4bace41f-402e-41be-90b9-64991aff4784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:35.709026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:35.710023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.710479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.710922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.724913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:35.726768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:35.727176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:35.751281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:35.752341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 463/902] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:06:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:35.762304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.803851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 440/903] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:35.807041 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bace41f-402e-41be-90b9-64991aff4784 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.899495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ec027-0a1f-41bc-ab9c-19530f02e773 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:35.919999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:35.946406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:35.946406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:35.946406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:35.946961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:35.947742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:35.948183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:35.950117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:35.956600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 464/904] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:35.957612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:35.975471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bace41f-402e-41be-90b9-64991aff4784 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.047718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.048886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.049337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.049780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:36.051158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:36.051621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:36.053909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:36.090871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 441/905] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Jun 24 09:06:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:36.090871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:36.103359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fa415-8737-47cd-bd6b-b554d69a6281 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.127909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:36.144680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.145611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.146019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.146486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:36.147619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:36.149753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:36.162927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:36.164809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:36.166293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 465/906] 10.222.0.52 () {56 vars in 1132 bytes} [Fri Jun 24 09:06:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:36.167680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:36.222303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:36.258166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc42d4-dcc8-46c2-be5f-33de07562b72 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.307205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.308154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.308437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.308747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:36.309730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:36.310271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:36.312846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:36.314932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00131ec3-d6d1-4e7b-97de-60af1ae3d618 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.319431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 442/907] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Jun 24 09:06:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:36.320349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:36.361450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.362467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.362958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.363414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:36.432068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:36.451779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd835ef-c00b-419e-adf2-6769e84734ae None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.453653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:36.455116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:36.495005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.495672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.495951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.496219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:36.496966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:36.497267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:36.498631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:36.503809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 443/908] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:36.504618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:36.518041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:36.610148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 466/909] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:36 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:06:36.610148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:36.655227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca2ab8-7522-436b-9ca0-87807038df25 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.703101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.705508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.705508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.705948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:36.706729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:36.708262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:36.740586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 444/910] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:06:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:36.740586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:36.756114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:36.767275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f645a4f-cba9-4c53-8b82-b2769ca0e669 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.817715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.818389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.818746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.819179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:36.820012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:36.820320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:36.822435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:36.834305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 467/911] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:36.835375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:36.841776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:36.942537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d7678-0464-478f-880e-01f76c5304b0 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.975970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a36b-0ee6-487a-ac58-1be4d073f77d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:36.991490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:36.992304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:36.992638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:36.992926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.017788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:37.021665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:37.044581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:37.044581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.044581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.044581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.045040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] RBAC: Adding query filter params (interface=public, service_id=c5f71d3ec0334054b803d31544978f38, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:37.045040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:37.045957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:37.051568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:37.052767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:37.053842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:37.053842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:37.053842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:37.053842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:37.054078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 468/912] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:06:36 2022] GET /identity/v3/endpoints?service_id=c5f71d3ec0334054b803d31544978f38&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:37.054910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 445/913] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:37.058614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:37.060516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:37.141243 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:37.149347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:37.201223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5ce0c-20a5-4365-9889-b3084284f77c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:37.217474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156a638-4791-4d49-8047-649d2902e934 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:37.244375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:37.245138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.245423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.245701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.246392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:37.247779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:37.260926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 469/914] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:37 2022] GET /identity/v3/services => generated 1497 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:06:37.286202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:37.287767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.288473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.289215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.290724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:37.291511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:37.294284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:37.318131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 446/915] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:37.327253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:37.466813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684b6c3-2df7-4785-86cd-9655526ee068 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:37.505272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:37.512243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:37.513527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.514240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.515010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.525592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:37.527318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:37.543715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:37.544672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:37.545568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:37.548249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 470/916] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:37.557087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:37.740279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b14fb5-b406-4d4e-8c58-c582d65aefd7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:37.772220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:37.801228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7859a42-361f-4554-ab5b-b126fb6ac8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:37.802476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7859a42-361f-4554-ab5b-b126fb6ac8ed None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.803117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7859a42-361f-4554-ab5b-b126fb6ac8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.803697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7859a42-361f-4554-ab5b-b126fb6ac8ed None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.804756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7859a42-361f-4554-ab5b-b126fb6ac8ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:37.805383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7859a42-361f-4554-ab5b-b126fb6ac8ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:37.807019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7859a42-361f-4554-ab5b-b126fb6ac8ed None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:37.821492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 447/917] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:37.831585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:37.860076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eff88-76a1-4a5c-a4b9-a46b61b3e01e None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.860532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eff88-76a1-4a5c-a4b9-a46b61b3e01e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.860978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eff88-76a1-4a5c-a4b9-a46b61b3e01e None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.862844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 448/918] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:37.872269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed020ddc-c16a-4afd-b753-8aafcc434ead None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.873147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed020ddc-c16a-4afd-b753-8aafcc434ead None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.873917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed020ddc-c16a-4afd-b753-8aafcc434ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.912875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ed020ddc-c16a-4afd-b753-8aafcc434ead None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:37.916531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ed020ddc-c16a-4afd-b753-8aafcc434ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:37.952443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d6813-bf22-4add-871f-c497fb4381ea None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:37.995209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:37.995845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:37.996129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:37.996408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:37.997292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] RBAC: Adding query filter params (role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=b2370d988bdf4499a1a3e71e8336d002) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:37.997602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:37.999041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:38.006825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 471/919] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:37 2022] GET /identity/v3/role_assignments?user.id=b2370d988bdf4499a1a3e71e8336d002&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:38.068750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 449/920] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:38.098023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:38.201183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:38.282725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa11c43-9088-4de5-8fea-743d34860e55 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:38.344311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:38.346061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:38.346777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:38.347516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:38.348957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:38.351703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:38.376156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:38.378103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:38.379694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:38.388679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 472/921] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:38.393909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed020ddc-c16a-4afd-b753-8aafcc434ead None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:38.541145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed020ddc-c16a-4afd-b753-8aafcc434ead None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:38.551973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:38.579760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:38.580433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:38.580734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:38.581011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:38.581728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:38.582035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:38.583414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:38.589454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 450/922] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:38.595163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:38.719591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81f07c-73d4-46dd-979a-54ea84ef1c8a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:38.741183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:38.757304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:38.757970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:38.758272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:38.758525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:38.759251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:38.759552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:38.761015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:38.766948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 473/923] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:38.772436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:38.883813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7ef68-6ed4-447e-a57f-a58665eaa40c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:38.923249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61724762-456b-4c51-9368-a4a18f581d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:38.923923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61724762-456b-4c51-9368-a4a18f581d68 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:38.924189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61724762-456b-4c51-9368-a4a18f581d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:38.924468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61724762-456b-4c51-9368-a4a18f581d68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:38.925298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61724762-456b-4c51-9368-a4a18f581d68 None admin] RBAC: Adding query filter params (interface=public, service_id=d18adc31274b4d57b41dae62fe0f244e, region_id=RegionOne) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:38.925616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61724762-456b-4c51-9368-a4a18f581d68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:38.927087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61724762-456b-4c51-9368-a4a18f581d68 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:38.936031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 451/924] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:06:38 2022] GET /identity/v3/endpoints?service_id=d18adc31274b4d57b41dae62fe0f244e&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:38.941644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:39.054603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1776d2-a74c-4052-9744-b69f8311cc01 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:39.091004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bfe7e-2451-4f96-935f-2ec577fb506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:39.091675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bfe7e-2451-4f96-935f-2ec577fb506c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:39.091935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bfe7e-2451-4f96-935f-2ec577fb506c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:39.092190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bfe7e-2451-4f96-935f-2ec577fb506c None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:39.092904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bfe7e-2451-4f96-935f-2ec577fb506c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:39.094266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bfe7e-2451-4f96-935f-2ec577fb506c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:39.100958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 474/925] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:38 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:06:39.489508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bd7b8-d5b7-4aac-aad5-f27f980f3df6 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:39.489819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bd7b8-d5b7-4aac-aad5-f27f980f3df6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:39.490065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bd7b8-d5b7-4aac-aad5-f27f980f3df6 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:39.491722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 452/926] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:39.497822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1ae1-1ede-4e58-9ac8-a350590fcea1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:39.498119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1ae1-1ede-4e58-9ac8-a350590fcea1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:39.498406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1ae1-1ede-4e58-9ac8-a350590fcea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:39.519073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7b2e1ae1-1ede-4e58-9ac8-a350590fcea1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:39.520261 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7b2e1ae1-1ede-4e58-9ac8-a350590fcea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:39.593101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:39.619627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 475/927] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:39.669494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bd7b8-d5b7-4aac-aad5-f27f980f3df6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:39.731131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9ecc-c9d7-4c76-83f1-6b77babf7a54 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:39.731606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9ecc-c9d7-4c76-83f1-6b77babf7a54 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:39.732074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9ecc-c9d7-4c76-83f1-6b77babf7a54 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:39.734695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:39.743493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 476/928] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:39.746295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8422ab-ccfd-4ce0-baac-2c5afb36f424 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:39.747681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8422ab-ccfd-4ce0-baac-2c5afb36f424 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:39.748133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8422ab-ccfd-4ce0-baac-2c5afb36f424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:39.781782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ce8422ab-ccfd-4ce0-baac-2c5afb36f424 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:39.783737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ce8422ab-ccfd-4ce0-baac-2c5afb36f424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:39.946929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bd7b8-d5b7-4aac-aad5-f27f980f3df6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:39.949324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 477/929] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:39.974854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8422ab-ccfd-4ce0-baac-2c5afb36f424 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:40.006559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:40.007219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:40.007568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:40.007926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:40.009039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:40.011121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:40.012644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING keystone.server.flask.common [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 09:06:40.038444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.sql.core [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 09:06:40.038444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 09:06:40.038444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [parameters: {'id': 'c125c7b48a1d47cd802d0764c82a9477', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 09:06:40.038444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80156) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c125c7b48a1d47cd802d0764c82a9477.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c125c7b48a1d47cd802d0764c82a9477. Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:06:40.049693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result.read() Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 09:06:40.050644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application yield resource Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise self.value Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application session.commit() Jun 24 09:06:40.051785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.session.flush() Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application compat.raise_( Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 09:06:40.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 09:06:40.053858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result.read() Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:06:40.054767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [parameters: {'id': 'c125c7b48a1d47cd802d0764c82a9477', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:40.056348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:40.058049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c125c7b48a1d47cd802d0764c82a9477. Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.065966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 453/930] 10.222.0.52 () {60 vars in 1140 bytes} [Fri Jun 24 09:06:39 2022] POST /identity/v3/projects => generated 262 bytes in 390 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:40.069766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:40.193466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:40.207314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855528b2-284d-4283-99be-e7487848c0e3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:40.234095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8422ab-ccfd-4ce0-baac-2c5afb36f424 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:40.250366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:40.251563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:40.252080 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:40.252654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:40.257824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:40.261164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:40.263114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:40.281669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:40.282639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:40.283605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:40.291421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 454/931] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:40.292517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:40.312630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:40.315033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:40.315549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:40.316053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:40.317955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:40.321181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:40.348623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:40.349708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:06:40.350582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:40.359175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 478/932] 10.222.0.52 () {56 vars in 1132 bytes} [Fri Jun 24 09:06:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:40.360375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:40.463364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d98228-c7e4-4837-afd2-f74ef3f26e73 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:40.502097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:40.502936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:40.503321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:40.503720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:40.504596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:40.505016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:40.506567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:40.525796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53806ea1-6cb4-4bc3-b6a5-9886f8cd620b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:40.534722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 455/933] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:40.596041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:40.597221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:40.597727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:40.598220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:40.599511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:40.600053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:40.602822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:40.613079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 479/934] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:40.618618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:40.636242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:40.736230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d9869-1072-4bae-8ad7-90f8c0953c25 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:40.775335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6cd4c-6d17-4d41-b8c3-08c1e02b007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:40.776084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6cd4c-6d17-4d41-b8c3-08c1e02b007d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:40.776345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6cd4c-6d17-4d41-b8c3-08c1e02b007d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:40.776649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6cd4c-6d17-4d41-b8c3-08c1e02b007d None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:40.777372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6cd4c-6d17-4d41-b8c3-08c1e02b007d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:40.777679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6cd4c-6d17-4d41-b8c3-08c1e02b007d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:40.779050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6cd4c-6d17-4d41-b8c3-08c1e02b007d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:40.785298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 456/935] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:40.794875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:40.862954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:40.979536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328868a4-bacc-47ff-bb4b-b0c87e540ef3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:41.042102 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed48403-1abd-44b1-981a-1ff717383b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:41.043196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48403-1abd-44b1-981a-1ff717383b6b None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.043693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48403-1abd-44b1-981a-1ff717383b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.044181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48403-1abd-44b1-981a-1ff717383b6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.045597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48403-1abd-44b1-981a-1ff717383b6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:41.048122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48403-1abd-44b1-981a-1ff717383b6b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:41.085355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 480/936] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:06:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:41.406220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301e35-ae9a-405d-85e7-ed3664cf53e3 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.407065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301e35-ae9a-405d-85e7-ed3664cf53e3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.407506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301e35-ae9a-405d-85e7-ed3664cf53e3 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.409348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 457/937] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:41.415731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d817-e521-4715-b234-1c6ce4b7388a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.416352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d817-e521-4715-b234-1c6ce4b7388a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.416874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d817-e521-4715-b234-1c6ce4b7388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.434903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-86a6d817-e521-4715-b234-1c6ce4b7388a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:41.436438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-86a6d817-e521-4715-b234-1c6ce4b7388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:41.449170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17747ed0-04cc-4b64-b44d-f55173fe25cd None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.449799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17747ed0-04cc-4b64-b44d-f55173fe25cd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.450316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17747ed0-04cc-4b64-b44d-f55173fe25cd None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.452158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 458/938] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:41.457689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb973038-b4dc-4f72-8fce-800eabd48566 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.458248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb973038-b4dc-4f72-8fce-800eabd48566 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.458735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb973038-b4dc-4f72-8fce-800eabd48566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.479564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-fb973038-b4dc-4f72-8fce-800eabd48566 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:41.481884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-fb973038-b4dc-4f72-8fce-800eabd48566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:41.520883 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 481/939] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:41.538889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6d817-e521-4715-b234-1c6ce4b7388a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:41.559670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:41.569087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 459/940] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:41.591859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb973038-b4dc-4f72-8fce-800eabd48566 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:41.599892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:41.669525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6d817-e521-4715-b234-1c6ce4b7388a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:41.698761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb973038-b4dc-4f72-8fce-800eabd48566 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:41.713628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:41.714824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.715109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.715389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.716317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:41.717893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:41.733982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:41.734998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:41.736414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 482/941] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:41.740975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:41.741805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.742085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.742357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.745718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:41.760023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:41.769480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:41.770456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:41.771303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:41.779458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 460/942] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Jun 24 09:06:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:41.780924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:41.836962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:41.855027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:41.861852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81580b60-af82-4c35-a521-e066facfd8a9 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:41.896646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368baa3-a55f-4ef0-ba79-868e845b9198 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:41.905596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:41.906286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.906573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.906852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.908100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:41.908416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:41.909846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:41.918146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 483/943] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:41.923660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:41.950977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:41.952140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:41.952677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:41.953202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:41.954191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:41.954692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:41.956452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:41.966816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 461/944] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Jun 24 09:06:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:41.973684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:42.051658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797dba5-eb3c-4d76-84d1-22398ec39022 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:42.091635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:42.092734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.093205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.093679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.094568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:42.095080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:42.096689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:42.100347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a80ac6-d5f2-4882-a61b-9501561bd487 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:42.107139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 484/945] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:42.108386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:42.123927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:42.133733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:42.143544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:42.144714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.145250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.145916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.159939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:42.161753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:42.180005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:42.181015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:42.181908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:42.181908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:42.181908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:42.181908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:42.188952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 462/946] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:42.190426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:42.240084 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863d5d2-1103-4e9a-9629-6be94e563462 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:42.278009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:42.279137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.279665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.280293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.281339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:42.283011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:42.312792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 485/947] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:06:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:42.325650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f59132-91d9-4ee9-95bd-01df964e981a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:42.374348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:42.375077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.375363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.375685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.376419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:42.376756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:42.378152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:42.393445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 463/948] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:42.401809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:42.478242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:42.516697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a2486-d06e-4f18-b56b-408c05976870 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:42.553229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:42.554110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.554398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.554728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.563908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:42.565387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:42.581453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:42.582462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:42.583835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 486/949] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:06:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:42.589345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:42.630020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:42.701018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86020d-a1d3-40da-9ad9-999bfe27fa64 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:42.739940 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3766c-6e58-49d7-a5e6-8e344dc23210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:42.740891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3766c-6e58-49d7-a5e6-8e344dc23210 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.741325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3766c-6e58-49d7-a5e6-8e344dc23210 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.741786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3766c-6e58-49d7-a5e6-8e344dc23210 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.742817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3766c-6e58-49d7-a5e6-8e344dc23210 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:42.743287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3766c-6e58-49d7-a5e6-8e344dc23210 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:42.744906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3766c-6e58-49d7-a5e6-8e344dc23210 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:42.764894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 464/950] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Jun 24 09:06:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:42.770759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:42.879495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079fc4-54e8-46ab-a40f-f41b98fa252e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:42.885165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1152a1-cdcf-422b-976b-17363b26ec7c None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.885455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1152a1-cdcf-422b-976b-17363b26ec7c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.885735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1152a1-cdcf-422b-976b-17363b26ec7c None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.887385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 465/951] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:42.892630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6b7c-345e-45b2-a367-131c6a834be4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.892809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6b7c-345e-45b2-a367-131c6a834be4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.893089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6b7c-345e-45b2-a367-131c6a834be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.910644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a4fa6b7c-345e-45b2-a367-131c6a834be4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:42.911837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a4fa6b7c-345e-45b2-a367-131c6a834be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:42.922967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:42.924589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:42.925287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:42.925721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:42.929816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:42.942980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:42.944427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:42.967988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:42.968993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:42.969881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:42.969881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:42.969881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:42.969881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:42.974394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 487/952] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:42.974777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.009031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 466/953] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:43.031458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6b7c-345e-45b2-a367-131c6a834be4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.067870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:43.111150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d4d5-8f4b-4fc0-9f5e-a34c9b5af83e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:43.150266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:43.150906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.151195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.151469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.152376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:43.152623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:43.154139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:43.172327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 488/954] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:43.173297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.174878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6b7c-345e-45b2-a367-131c6a834be4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:43.218718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:43.223496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:43.224510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.225009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.225765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.227753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:43.229660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:43.248098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.sql.core [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 09:06:43.248098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 09:06:43.248098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [parameters: {'user_id': '191ef876cc6a4957a36174075cff3181', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 09:06:43.248098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80156) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result.read() Jun 24 09:06:43.259593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:06:43.260545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application yield resource Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise self.value Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application session.commit() Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 09:06:43.262509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.session.flush() Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application compat.raise_( Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 09:06:43.265743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:06:43.269998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application result.read() Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:06:43.274339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [parameters: {'user_id': '191ef876cc6a4957a36174075cff3181', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:43.279990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 09:06:43.283447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 467/955] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Jun 24 09:06:43 2022] POST /identity/v3/users => generated 160 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:43.288134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.312054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490959-0e04-4090-a4f2-65ca7090eec8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:43.352894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8954936-5991-4ab8-9b3f-9da851152269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:43.353930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8954936-5991-4ab8-9b3f-9da851152269 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.354128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8954936-5991-4ab8-9b3f-9da851152269 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.354444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8954936-5991-4ab8-9b3f-9da851152269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.362995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8954936-5991-4ab8-9b3f-9da851152269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:43.364485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8954936-5991-4ab8-9b3f-9da851152269 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-a8954936-5991-4ab8-9b3f-9da851152269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:43.379666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:43.380857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:43.382045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:43.384713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 489/956] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:43.387881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe5311-b204-49a4-8775-8b6dde4b50a5 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.388461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe5311-b204-49a4-8775-8b6dde4b50a5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.388616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe5311-b204-49a4-8775-8b6dde4b50a5 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.392615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 490/957] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:43.392615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe5311-b204-49a4-8775-8b6dde4b50a5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.398035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:43.415823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171ce9e-45c0-4a29-bf86-ec4e10d29a8b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:43.453711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccbf08-ce2f-41a5-903c-09efc69f762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:43.454519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccbf08-ce2f-41a5-903c-09efc69f762a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.454799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccbf08-ce2f-41a5-903c-09efc69f762a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.455073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccbf08-ce2f-41a5-903c-09efc69f762a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.463790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccbf08-ce2f-41a5-903c-09efc69f762a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:43.465468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccbf08-ce2f-41a5-903c-09efc69f762a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-e8ccbf08-ce2f-41a5-903c-09efc69f762a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:43.480091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:43.482171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:06:43.484251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:43.486123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 468/958] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:43.487581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e7608-b485-4cc7-b994-a2ffa0ece074 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.487883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e7608-b485-4cc7-b994-a2ffa0ece074 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.488158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e7608-b485-4cc7-b994-a2ffa0ece074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.506186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-0e2e7608-b485-4cc7-b994-a2ffa0ece074 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:43.507724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0e2e7608-b485-4cc7-b994-a2ffa0ece074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:43.509492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:43.517389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe5311-b204-49a4-8775-8b6dde4b50a5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:43.559391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:43.560149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.560501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.560882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.561677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:43.561980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:43.563400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:43.577767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 491/959] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:43.578360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.590075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 469/960] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:43.591052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e7608-b485-4cc7-b994-a2ffa0ece074 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.684115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:43.715619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e7608-b485-4cc7-b994-a2ffa0ece074 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:43.716483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eae84b-4202-4c72-9fc3-0429095dc6a8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:43.763339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:43.763717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:43.764656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.764803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:43.765007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.765236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/b2370d988bdf4499a1a3e71e8336d002/roles/54ceb2d228bc4fefb2dd6fcbc299a1ea` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.765674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:43.766148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:43.767178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:43.767714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=b2370d988bdf4499a1a3e71e8336d002, role_id=54ceb2d228bc4fefb2dd6fcbc299a1ea)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:43.767959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:43.769244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:43.769763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:43.785790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 470/961] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:06:43 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/b2370d988bdf4499a1a3e71e8336d002/roles/54ceb2d228bc4fefb2dd6fcbc299a1ea => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:06:43.788836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:43.840352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 492/962] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Jun 24 09:06:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:43.978525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220365-08ef-4390-9ef9-d69c434b676d None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:44.043279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:44.044703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:44.045243 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:44.045765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:44.047045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:44.049510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:44.054301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:44.073827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:44.075153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:44.076172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:44.080132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 471/963] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:44.085558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:44.106762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:44.206656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a7d2f-1c48-4397-82fa-7bb0260d2bc8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:44.244315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddc07f-81f3-4e6d-8512-70c8287f4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:44.245023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc07f-81f3-4e6d-8512-70c8287f4988 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:44.245305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc07f-81f3-4e6d-8512-70c8287f4988 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:44.245583 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc07f-81f3-4e6d-8512-70c8287f4988 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:44.246289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddc07f-81f3-4e6d-8512-70c8287f4988 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:44.246592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddc07f-81f3-4e6d-8512-70c8287f4988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:44.247973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddc07f-81f3-4e6d-8512-70c8287f4988 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:44.253635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 493/964] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:44.260334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:44.369182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bda7de-2004-45c8-a744-ad938455eb10 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:44.405418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ae1b6-c3cf-4b6e-a53a-d5e03e3e44a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:44.406095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ae1b6-c3cf-4b6e-a53a-d5e03e3e44a0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:44.406400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ae1b6-c3cf-4b6e-a53a-d5e03e3e44a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:44.406648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ae1b6-c3cf-4b6e-a53a-d5e03e3e44a0 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:44.407357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ae1b6-c3cf-4b6e-a53a-d5e03e3e44a0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:44.407661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ae1b6-c3cf-4b6e-a53a-d5e03e3e44a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:44.409061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ae1b6-c3cf-4b6e-a53a-d5e03e3e44a0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:44.421323 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 472/965] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:44.485457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75518-a79a-46aa-9a4b-f8488ea50050 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:44.485943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75518-a79a-46aa-9a4b-f8488ea50050 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:44.486442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75518-a79a-46aa-9a4b-f8488ea50050 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:44.489232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 494/966] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:44.496622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d768c-9a78-4afa-845f-0a3e333afee8 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:44.497001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d768c-9a78-4afa-845f-0a3e333afee8 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:44.497259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d768c-9a78-4afa-845f-0a3e333afee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:44.515929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-531d768c-9a78-4afa-845f-0a3e333afee8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:44.517187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-531d768c-9a78-4afa-845f-0a3e333afee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:44.589434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:44.600015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 473/967] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:44.623854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75518-a79a-46aa-9a4b-f8488ea50050 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:44.756191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:44.781106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75518-a79a-46aa-9a4b-f8488ea50050 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:44.821217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:44.822180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:44.822567 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:44.822949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:44.823704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:44.825214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:44.838624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:44.840937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:44.842913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 495/968] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:44.850032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d768c-9a78-4afa-845f-0a3e333afee8 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:44.962059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d768c-9a78-4afa-845f-0a3e333afee8 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:45.001216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:45.001891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:45.002174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:45.002456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:45.003161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:45.003468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:45.004864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:45.010979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 474/969] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:45.019830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:45.185710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17875fd6-4512-4468-9b02-057689b6d374 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:45.242299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:45.244634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:45.245399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:45.246296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:45.247885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:45.248786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:45.251628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:45.267835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 496/970] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:45.269168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:45.336010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:45.385191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847df7-fb03-41f2-ab56-182049ea50c6 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:45.422664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d952e3e-4e58-46d2-9035-7ab36f5908a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:45.423324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952e3e-4e58-46d2-9035-7ab36f5908a2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:45.423601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952e3e-4e58-46d2-9035-7ab36f5908a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:45.423880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952e3e-4e58-46d2-9035-7ab36f5908a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:45.424717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d952e3e-4e58-46d2-9035-7ab36f5908a2 None admin] RBAC: Adding query filter params (interface=admin, service_id=d18adc31274b4d57b41dae62fe0f244e, region_id=RegionOne) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:45.425031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d952e3e-4e58-46d2-9035-7ab36f5908a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:45.426404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d952e3e-4e58-46d2-9035-7ab36f5908a2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:45.432766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 475/971] 10.222.0.52 () {56 vars in 1249 bytes} [Fri Jun 24 09:06:45 2022] GET /identity/v3/endpoints?service_id=d18adc31274b4d57b41dae62fe0f244e&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:45.439058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:45.456974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:45.590704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c17e5-57ee-4610-af00-5a4c281a33ac None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:45.640988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d7548-8db6-435d-8852-d522473f3b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:45.642042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d7548-8db6-435d-8852-d522473f3b26 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:45.642468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d7548-8db6-435d-8852-d522473f3b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:45.642885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d7548-8db6-435d-8852-d522473f3b26 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:45.643997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d7548-8db6-435d-8852-d522473f3b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:45.646209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d7548-8db6-435d-8852-d522473f3b26 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:45.665009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 497/972] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:45 2022] GET /identity/v3/services => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:06:45.997346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c648ce-e164-42a0-9b45-7198f35c7627 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:45.998004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c648ce-e164-42a0-9b45-7198f35c7627 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:45.998004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c648ce-e164-42a0-9b45-7198f35c7627 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:45.999367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 476/973] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:46.005356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4face7-c729-4ffe-b5c1-7c50d3328577 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:46.005879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4face7-c729-4ffe-b5c1-7c50d3328577 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:46.006375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4face7-c729-4ffe-b5c1-7c50d3328577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:46.031804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7a4face7-c729-4ffe-b5c1-7c50d3328577 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:46.035481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7a4face7-c729-4ffe-b5c1-7c50d3328577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:46.138858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 498/974] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:46.163420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c648ce-e164-42a0-9b45-7198f35c7627 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:46.318002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c648ce-e164-42a0-9b45-7198f35c7627 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:46.352112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:46.367083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2f012-308b-4769-8931-490248d37064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:46.368615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2f012-308b-4769-8931-490248d37064 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:46.368899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2f012-308b-4769-8931-490248d37064 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:46.369169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2f012-308b-4769-8931-490248d37064 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:46.380158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2f012-308b-4769-8931-490248d37064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-afc2f012-308b-4769-8931-490248d37064 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:46.388913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:46.389851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:46.390721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:46.392868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 477/975] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Jun 24 09:06:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:46.395917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4face7-c729-4ffe-b5c1-7c50d3328577 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:46.398113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:46.500790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4face7-c729-4ffe-b5c1-7c50d3328577 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:46.533460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81d07-b6ca-435b-90dc-13c77f975cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:46.534130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81d07-b6ca-435b-90dc-13c77f975cff None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:46.534412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81d07-b6ca-435b-90dc-13c77f975cff None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:46.534684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81d07-b6ca-435b-90dc-13c77f975cff None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:46.535422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81d07-b6ca-435b-90dc-13c77f975cff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:46.535724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81d07-b6ca-435b-90dc-13c77f975cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:46.537111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81d07-b6ca-435b-90dc-13c77f975cff None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:46.545817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 499/976] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Jun 24 09:06:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:46.551111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2f012-308b-4769-8931-490248d37064 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:46.656728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2f012-308b-4769-8931-490248d37064 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:46.699612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:46.700415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:46.700749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:46.701023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:46.712010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964afb-dc7f-41d4-a361-cdf131b19e1e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:46.712370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964afb-dc7f-41d4-a361-cdf131b19e1e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:46.712583 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964afb-dc7f-41d4-a361-cdf131b19e1e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:46.714877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:46.715934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:46.717923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 500/977] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:46.719134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb38295-90e9-484c-8acd-7dee269d42ac None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:46.719422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb38295-90e9-484c-8acd-7dee269d42ac None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:46.719691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb38295-90e9-484c-8acd-7dee269d42ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:46.734510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:46.736090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:46.737750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:46.737750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:46.737750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:46.737750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:46.737750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 478/978] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:46.739648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7cb38295-90e9-484c-8acd-7dee269d42ac None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:46.741008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7cb38295-90e9-484c-8acd-7dee269d42ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:46.741447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:46.790407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:46.820025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:46.833680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 501/979] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:46.872782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb38295-90e9-484c-8acd-7dee269d42ac None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:46.890709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b6f1e-1895-42f7-8477-b0a605f4d4d0 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:46.943429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:46.944389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:46.944852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:46.945308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:46.946524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:46.947029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:46.949410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:46.972890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 479/980] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:46.974415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:47.025266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb38295-90e9-484c-8acd-7dee269d42ac None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:47.066459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:47.067128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:47.067409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:47.067683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:47.068414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:47.069812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:47.088607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 502/981] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:46 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:06:47.121084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:47.129887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89e9dc-7968-4729-8e5d-b031a87ea3c4 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:47.180505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:47.181371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:47.181648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:47.181918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:47.191024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:47.192485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:47.208885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:47.210485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:06:47.213385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:47.214834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 480/982] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:06:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:47.216597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:47.286601 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:47.338453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe24a04-3bb1-4c03-864c-377708c62a29 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:47.375700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cb883-bc76-4292-a777-aa30ffa9bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:47.376388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb883-bc76-4292-a777-aa30ffa9bd7d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:47.376671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb883-bc76-4292-a777-aa30ffa9bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:47.376949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb883-bc76-4292-a777-aa30ffa9bd7d None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:47.377777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cb883-bc76-4292-a777-aa30ffa9bd7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:47.378079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cb883-bc76-4292-a777-aa30ffa9bd7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:47.379496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cb883-bc76-4292-a777-aa30ffa9bd7d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:47.400003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 503/983] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Jun 24 09:06:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:47.406144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:47.575320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac47134-bd46-4bbe-a2ce-f7c374962020 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:47.630675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:47.631762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:47.632248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:47.632764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:47.664583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:47.747564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:47.750128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:47.779839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac012dd8-5454-4a91-8b76-6798e9f24474 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:47.780119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac012dd8-5454-4a91-8b76-6798e9f24474 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:47.780391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac012dd8-5454-4a91-8b76-6798e9f24474 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:47.781798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 504/984] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:47.786808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce30dfe-b050-4123-b839-88e369553e80 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:47.787185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce30dfe-b050-4123-b839-88e369553e80 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:47.787654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce30dfe-b050-4123-b839-88e369553e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:47.812853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9ce30dfe-b050-4123-b839-88e369553e80 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:47.814801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9ce30dfe-b050-4123-b839-88e369553e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:47.870096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 481/985] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:47 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:06:47.875377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:47.927969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:47.936601 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 505/986] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:47.958874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce30dfe-b050-4123-b839-88e369553e80 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:47.996446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387de96-5fae-4f1e-9bc0-d3f6c728a2db None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.027082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:48.056435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.057552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.058270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.058741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.075627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.077621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:48.108050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:48.109075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:48.109989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:48.109989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:48.109989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:48.109989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:48.109989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 482/987] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:48.119099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:48.125781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce30dfe-b050-4123-b839-88e369553e80 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.173037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.173973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.174637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.174637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.175168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.176538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:48.191486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:48.193743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:06:48.195363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:48.198017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 506/988] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Jun 24 09:06:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:48.198649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:48.254250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579eeb8-d432-4305-8e98-9d2fe4bca6cd None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.256592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:48.296849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.297519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.297801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.298076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.298798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:48.299577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.300518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.317830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 483/989] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:48.325415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:48.330481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d384-5573-4d02-b0ae-a398f9a12a22 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.341589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:48.373611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.374288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.374571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.374846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.375619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:48.375976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.377387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.388172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 507/990] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:48.389423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:48.491665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f106e5-f092-428d-9fc4-7c315b2a824a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.505059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42078643-acae-474e-acc0-a8286fb10e14 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.532161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.533531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.534018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.534489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.546807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.549466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.552159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.552852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.553134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.553407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.554112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:48.554408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.555824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.561574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 508/991] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Jun 24 09:06:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:48.567143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:48.578130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:48.579819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:48.581549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 484/992] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:48.586856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:48.621066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:48.623247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:48.727323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd32f7-54a1-4996-9a2e-abf5b64e5192 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.742636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef957-04df-4040-a13f-c88d84cf31b5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:48.770838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.772041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.772591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.773039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.774734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.776795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.785513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16064f-3cfe-4039-823a-645f9ed98f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:48.786353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16064f-3cfe-4039-823a-645f9ed98f0d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:48.786857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16064f-3cfe-4039-823a-645f9ed98f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:48.787143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16064f-3cfe-4039-823a-645f9ed98f0d None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:48.788029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16064f-3cfe-4039-823a-645f9ed98f0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:48.788354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16064f-3cfe-4039-823a-645f9ed98f0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:48.790379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16064f-3cfe-4039-823a-645f9ed98f0d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:48.798079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 509/993] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:06:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:06:48.816201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 485/994] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:48.825670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:48.951388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:48.972526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f258725-4dce-4cbb-ae87-2cb791f26e73 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:49.018643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dae08-5378-44e6-9460-447199d77140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:49.023531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dae08-5378-44e6-9460-447199d77140 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:49.023531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dae08-5378-44e6-9460-447199d77140 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:49.023531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dae08-5378-44e6-9460-447199d77140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:49.023531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dae08-5378-44e6-9460-447199d77140 None admin] RBAC: Adding query filter params (role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=b2370d988bdf4499a1a3e71e8336d002) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:49.023531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dae08-5378-44e6-9460-447199d77140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:49.027302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115dae08-5378-44e6-9460-447199d77140 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:49.038907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 510/995] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:48 2022] GET /identity/v3/role_assignments?user.id=b2370d988bdf4499a1a3e71e8336d002&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:49.538074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18de9f-835a-4ea6-b9b4-2dce3c288b37 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:49.538629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18de9f-835a-4ea6-b9b4-2dce3c288b37 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:49.538629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18de9f-835a-4ea6-b9b4-2dce3c288b37 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:49.540784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 486/996] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:49.545952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35e3a-ca22-4fef-9c2f-cee56ed0b84f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:49.546400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35e3a-ca22-4fef-9c2f-cee56ed0b84f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:49.546574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35e3a-ca22-4fef-9c2f-cee56ed0b84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:49.565658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8ab35e3a-ca22-4fef-9c2f-cee56ed0b84f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:49.566840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8ab35e3a-ca22-4fef-9c2f-cee56ed0b84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:49.644775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 511/997] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:49.668329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18de9f-835a-4ea6-b9b4-2dce3c288b37 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:49.761511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:49.781572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18de9f-835a-4ea6-b9b4-2dce3c288b37 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:49.817383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:49.818209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:49.818489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:49.818760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:49.819406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:49.820883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:49.833058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:49.834212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:49.835058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:49.839594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 487/998] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:49.840916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35e3a-ca22-4fef-9c2f-cee56ed0b84f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:49.925576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:49.942821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35e3a-ca22-4fef-9c2f-cee56ed0b84f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:49.978331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:49.979042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:49.979323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:49.979644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:49.980352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:49.980677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:49.982031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:49.988940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 512/999] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:49.992829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:50.099567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212f03-f796-4cf6-a92d-d32b7e6b404a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:50.134509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:50.135185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:50.135466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:50.135744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:50.136446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:50.136782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:50.138149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:50.147865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 488/1000] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:50.149501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:50.252886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c7c8-1d62-44a2-bcf7-0bd476c9b43d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:50.288779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7746b-7228-4d15-9efb-cb2342a62722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:50.289387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7746b-7228-4d15-9efb-cb2342a62722 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:50.289670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7746b-7228-4d15-9efb-cb2342a62722 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:50.289944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7746b-7228-4d15-9efb-cb2342a62722 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:50.290733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7746b-7228-4d15-9efb-cb2342a62722 None admin] RBAC: Adding query filter params (interface=public, service_id=b3ba558bbacd4f0e8f5baccc092f4aea, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:50.291043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7746b-7228-4d15-9efb-cb2342a62722 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:50.292398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7746b-7228-4d15-9efb-cb2342a62722 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:50.303561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 513/1001] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:06:50 2022] GET /identity/v3/endpoints?service_id=b3ba558bbacd4f0e8f5baccc092f4aea&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:50.303736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:50.316856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:50.408660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23feb30-4c66-4211-8b91-5684fc0d782a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:50.441355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344611a-d179-40e8-a86d-67f84c9a557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:50.442015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344611a-d179-40e8-a86d-67f84c9a557b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:50.442294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344611a-d179-40e8-a86d-67f84c9a557b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:50.442566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344611a-d179-40e8-a86d-67f84c9a557b None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:50.443238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344611a-d179-40e8-a86d-67f84c9a557b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:50.444601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344611a-d179-40e8-a86d-67f84c9a557b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:50.455767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 489/1002] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:50 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:06:51.115921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa07f0-2ba3-477b-95b4-cae09f5bb2ee None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.116792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa07f0-2ba3-477b-95b4-cae09f5bb2ee None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.116792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa07f0-2ba3-477b-95b4-cae09f5bb2ee None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.118687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 514/1003] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:51.124597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0f62-345a-4ab6-b9af-43897a615e76 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.125244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0f62-345a-4ab6-b9af-43897a615e76 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.125395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0f62-345a-4ab6-b9af-43897a615e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.144643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9d1c0f62-345a-4ab6-b9af-43897a615e76 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:51.146409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9d1c0f62-345a-4ab6-b9af-43897a615e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:51.225564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 490/1004] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:51.251219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa07f0-2ba3-477b-95b4-cae09f5bb2ee None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:51.268745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:51.286471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136793e-b6c9-42d8-89c5-70c267c86d53 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.286794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136793e-b6c9-42d8-89c5-70c267c86d53 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.287024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136793e-b6c9-42d8-89c5-70c267c86d53 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.288801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 491/1005] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:51.295314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f4db3-4f1c-40c5-8123-7946eb29af42 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.295692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f4db3-4f1c-40c5-8123-7946eb29af42 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.296069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f4db3-4f1c-40c5-8123-7946eb29af42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.303011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:51.316529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d10f4db3-4f1c-40c5-8123-7946eb29af42 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:51.317767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d10f4db3-4f1c-40c5-8123-7946eb29af42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:51.370645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa07f0-2ba3-477b-95b4-cae09f5bb2ee None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:51.393057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 492/1006] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:51.412498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:51.413320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.413602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.413908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.415640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:51.417254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:51.417905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f4db3-4f1c-40c5-8123-7946eb29af42 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:51.419007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 515/1007] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:51.424491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:51.525780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:51.532360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f4db3-4f1c-40c5-8123-7946eb29af42 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:51.553444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b339b-7fb9-4d19-b4f2-5c99d1621566 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:51.572834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:51.573992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.574261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.574885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] PATH_INFO: `/v3/services/image` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.575179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:51.576589 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:51.578847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:51.593883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:51.595122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:06:51.595994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:51.596779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 493/1008] 10.222.0.52 () {56 vars in 1105 bytes} [Fri Jun 24 09:06:51 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:51.603037 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:51.613155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:51.614165 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.614592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.615019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.616151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:51.618612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:51.672594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 516/1009] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:51 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:06:51.759111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed61627-da52-4618-bc4c-0afc5c998961 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:51.803198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc2d94-e04a-4db8-82b2-0cab2acc3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:51.803943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2d94-e04a-4db8-82b2-0cab2acc3b8e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.804171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2d94-e04a-4db8-82b2-0cab2acc3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.804452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2d94-e04a-4db8-82b2-0cab2acc3b8e None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.805258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc2d94-e04a-4db8-82b2-0cab2acc3b8e None admin] RBAC: Adding query filter params (name=image) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:51.805569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc2d94-e04a-4db8-82b2-0cab2acc3b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:51.806956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc2d94-e04a-4db8-82b2-0cab2acc3b8e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:51.813305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 494/1010] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:51.819390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:51.949908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0219e0f-da3b-44ab-8890-8ee2b2b02f02 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:51.986588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:51.994728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa7e1a-e234-4e17-9fd4-bfc853d898aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:51.995565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa7e1a-e234-4e17-9fd4-bfc853d898aa None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:51.995928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa7e1a-e234-4e17-9fd4-bfc853d898aa None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:51.996330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa7e1a-e234-4e17-9fd4-bfc853d898aa None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:51.997592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa7e1a-e234-4e17-9fd4-bfc853d898aa None admin] RBAC: Adding query filter params (type=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:51.998152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa7e1a-e234-4e17-9fd4-bfc853d898aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:51.999748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa7e1a-e234-4e17-9fd4-bfc853d898aa None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:52.006738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 517/1011] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:52.866613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6217302-9f18-4078-92d2-973d17a89275 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:52.867195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6217302-9f18-4078-92d2-973d17a89275 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:52.867604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6217302-9f18-4078-92d2-973d17a89275 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:52.870200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 495/1012] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:52.877111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528c7b-2694-4890-90cf-f9939d3daf7f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:52.877804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528c7b-2694-4890-90cf-f9939d3daf7f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:52.877804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528c7b-2694-4890-90cf-f9939d3daf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:52.896173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3d528c7b-2694-4890-90cf-f9939d3daf7f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:52.897442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3d528c7b-2694-4890-90cf-f9939d3daf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:52.974641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 518/1013] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:52.997807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6217302-9f18-4078-92d2-973d17a89275 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:53.109454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:53.177935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6217302-9f18-4078-92d2-973d17a89275 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:53.234408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:53.235760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:53.236249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:53.236768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:53.237954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:53.240437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:53.263145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:53.264195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:53.270481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 496/1014] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:53.271388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d528c7b-2694-4890-90cf-f9939d3daf7f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:53.375605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d528c7b-2694-4890-90cf-f9939d3daf7f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:53.404979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:53.415203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:53.415885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:53.416175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:53.416451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:53.417198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:53.417507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:53.418878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:53.429609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 519/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:53.431061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:53.537371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b2d2d-6604-4749-a415-eeb44b87a07f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:53.564390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:53.575011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44a86f-8772-4aba-ada4-75aa80b31d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:53.575696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44a86f-8772-4aba-ada4-75aa80b31d28 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:53.575979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44a86f-8772-4aba-ada4-75aa80b31d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:53.576255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44a86f-8772-4aba-ada4-75aa80b31d28 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:53.577012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44a86f-8772-4aba-ada4-75aa80b31d28 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:53.577315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44a86f-8772-4aba-ada4-75aa80b31d28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:53.578682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44a86f-8772-4aba-ada4-75aa80b31d28 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:53.584226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 497/1016] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Jun 24 09:06:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:53.590389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:53.693122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1080603-17b5-40d0-98b9-350dac13c5d5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:53.727761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777a606-5c31-4d69-9107-94dfb7cc9e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:53.728426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a606-5c31-4d69-9107-94dfb7cc9e91 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:53.728746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a606-5c31-4d69-9107-94dfb7cc9e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:53.729020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a606-5c31-4d69-9107-94dfb7cc9e91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:53.729771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777a606-5c31-4d69-9107-94dfb7cc9e91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:53.731131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777a606-5c31-4d69-9107-94dfb7cc9e91 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:53.751769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 520/1017] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:06:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:53.843387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31ee5f-11fa-4901-bb1f-ce1c3e67bb4f None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:53.843723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31ee5f-11fa-4901-bb1f-ce1c3e67bb4f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:53.843944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31ee5f-11fa-4901-bb1f-ce1c3e67bb4f None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:53.845568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 498/1018] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:53.851222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b69ac-e439-42d6-9bf5-ea8b7b812e55 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:53.851510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b69ac-e439-42d6-9bf5-ea8b7b812e55 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:53.851833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b69ac-e439-42d6-9bf5-ea8b7b812e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:53.870003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-478b69ac-e439-42d6-9bf5-ea8b7b812e55 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:53.871225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-478b69ac-e439-42d6-9bf5-ea8b7b812e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:53.920538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:53.958682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 521/1019] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:53.987776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31ee5f-11fa-4901-bb1f-ce1c3e67bb4f None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:54.121992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31ee5f-11fa-4901-bb1f-ce1c3e67bb4f None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:54.158850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240ac8e-b50b-48a9-8bc6-3e9c4c614bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:54.159653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240ac8e-b50b-48a9-8bc6-3e9c4c614bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:54.159992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240ac8e-b50b-48a9-8bc6-3e9c4c614bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:54.160276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240ac8e-b50b-48a9-8bc6-3e9c4c614bc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:54.161528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240ac8e-b50b-48a9-8bc6-3e9c4c614bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:54.162967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240ac8e-b50b-48a9-8bc6-3e9c4c614bc9 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:54.165405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 499/1020] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:54.171509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b69ac-e439-42d6-9bf5-ea8b7b812e55 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:54.321628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b69ac-e439-42d6-9bf5-ea8b7b812e55 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:54.360150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853c24-7537-46cb-bf0d-f42bde9e33da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:54.360863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853c24-7537-46cb-bf0d-f42bde9e33da None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:54.361142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853c24-7537-46cb-bf0d-f42bde9e33da None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:54.361417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853c24-7537-46cb-bf0d-f42bde9e33da None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:54.363666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853c24-7537-46cb-bf0d-f42bde9e33da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:54.363666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853c24-7537-46cb-bf0d-f42bde9e33da None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:54.393149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 522/1021] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:06:54 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:06:54.513872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57131c68-9560-405f-a986-9899db1113a5 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:54.514281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57131c68-9560-405f-a986-9899db1113a5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:54.514493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57131c68-9560-405f-a986-9899db1113a5 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:54.516123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 500/1022] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:54.522907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21756b-0611-4260-8745-2d5a6998a0ff None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:54.523940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21756b-0611-4260-8745-2d5a6998a0ff None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:54.527109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21756b-0611-4260-8745-2d5a6998a0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:54.555467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ba21756b-0611-4260-8745-2d5a6998a0ff None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:54.557517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ba21756b-0611-4260-8745-2d5a6998a0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:54.598358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:54.657443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 523/1023] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:54 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:54.680592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57131c68-9560-405f-a986-9899db1113a5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:54.743880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:54.797835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57131c68-9560-405f-a986-9899db1113a5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:54.833206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd030208-c390-41e8-8d42-7235d4ad1b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:54.833877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd030208-c390-41e8-8d42-7235d4ad1b8d None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:54.834186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd030208-c390-41e8-8d42-7235d4ad1b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:54.834467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd030208-c390-41e8-8d42-7235d4ad1b8d None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:54.835204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd030208-c390-41e8-8d42-7235d4ad1b8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:54.837166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd030208-c390-41e8-8d42-7235d4ad1b8d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:54.851405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 501/1024] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:06:54 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:06:56.918246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac863-f532-41d0-80ff-8678a770ae28 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:56.918897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac863-f532-41d0-80ff-8678a770ae28 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:56.919253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac863-f532-41d0-80ff-8678a770ae28 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:56.921703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 524/1025] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:06:56.927857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c706b9-55f5-4c2d-ac27-142df23d7314 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:56.928578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c706b9-55f5-4c2d-ac27-142df23d7314 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:56.928578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c706b9-55f5-4c2d-ac27-142df23d7314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:56.949993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9069-95b8-4443-9050-ff78edf89488 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:56.950301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9069-95b8-4443-9050-ff78edf89488 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:56.950572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9069-95b8-4443-9050-ff78edf89488 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:56.952069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 525/1026] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:56.954653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-62c706b9-55f5-4c2d-ac27-142df23d7314 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:56.956771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-62c706b9-55f5-4c2d-ac27-142df23d7314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:56.956971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bab5a4-ff33-4d38-b8a7-bef7aed56f99 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:56.957196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bab5a4-ff33-4d38-b8a7-bef7aed56f99 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:56.957480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bab5a4-ff33-4d38-b8a7-bef7aed56f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:56.974395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-18bab5a4-ff33-4d38-b8a7-bef7aed56f99 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:56.975337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-18bab5a4-ff33-4d38-b8a7-bef7aed56f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:57.063916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 526/1027] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:57.080605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 502/1028] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:06:57.086726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bab5a4-ff33-4d38-b8a7-bef7aed56f99 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.103406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c706b9-55f5-4c2d-ac27-142df23d7314 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.159861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.206733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.220150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c706b9-55f5-4c2d-ac27-142df23d7314 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.232070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bab5a4-ff33-4d38-b8a7-bef7aed56f99 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.265991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.267240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.267739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.268092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.273745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.274542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.274822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.275096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.275758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:57.277163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:57.284230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:57.288869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:57.290387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:57.296666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 527/1029] 10.222.0.52 () {56 vars in 1105 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/services/image => generated 86 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:57.297522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:57.300045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:57.301515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:57.302703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 503/1030] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:57.308638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.431721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1bdb-2d56-4b93-b989-15f9b377fcb3 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.441036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e282e-9768-4e39-b0a6-b45791c3de64 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.452440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.473809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdfa12-2242-4605-bda9-6864445e5c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.474497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdfa12-2242-4605-bda9-6864445e5c38 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.474713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdfa12-2242-4605-bda9-6864445e5c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.474950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdfa12-2242-4605-bda9-6864445e5c38 None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.475656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdfa12-2242-4605-bda9-6864445e5c38 None admin] RBAC: Adding query filter params (name=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:57.475971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdfa12-2242-4605-bda9-6864445e5c38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:57.477932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdfa12-2242-4605-bda9-6864445e5c38 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:57.482629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae2144-6c04-40fd-b8e6-61ec7f62122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.483316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae2144-6c04-40fd-b8e6-61ec7f62122e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.483596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae2144-6c04-40fd-b8e6-61ec7f62122e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.483875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae2144-6c04-40fd-b8e6-61ec7f62122e None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.489795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae2144-6c04-40fd-b8e6-61ec7f62122e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:57.489977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 528/1031] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:57.490125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae2144-6c04-40fd-b8e6-61ec7f62122e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:57.490125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae2144-6c04-40fd-b8e6-61ec7f62122e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:57.492393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1302e-5c9f-42dc-aee1-10e48fc3cdb5 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.492393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1302e-5c9f-42dc-aee1-10e48fc3cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.492393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1302e-5c9f-42dc-aee1-10e48fc3cdb5 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.493447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 529/1032] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:06:57.494137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1302e-5c9f-42dc-aee1-10e48fc3cdb5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.502791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 504/1033] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:57.506239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab881e0-1fb8-4fd8-8ddd-b1befac4689e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.506684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab881e0-1fb8-4fd8-8ddd-b1befac4689e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.507578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab881e0-1fb8-4fd8-8ddd-b1befac4689e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.510157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.530234 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-0ab881e0-1fb8-4fd8-8ddd-b1befac4689e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:06:57.531439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0ab881e0-1fb8-4fd8-8ddd-b1befac4689e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:06:57.610476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 505/1034] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:06:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:06:57.611342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab881e0-1fb8-4fd8-8ddd-b1befac4689e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.612184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1302e-5c9f-42dc-aee1-10e48fc3cdb5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.652396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.653105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.653393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.653674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.654382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] RBAC: Adding query filter params (type=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:57.654692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:57.656066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:57.661367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 530/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:57.662261 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.736365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.739199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab881e0-1fb8-4fd8-8ddd-b1befac4689e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.749603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.775574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c80a9-18ce-4e52-96c4-e2acb6f336ce None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.778824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.779738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.780073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.780401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.791124 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:57.792536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:57.811195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:57.812523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:57.813790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:57.813790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:57.813790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:57.813790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:57.813790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 506/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:57.813790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.819063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.820062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.820329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.820624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.831109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:06:57.838674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:57.839688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:06:57.840578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 531/1037] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:57.841315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.917856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0e49-b35e-464c-87ea-75fbfb22c031 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.942302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5826e-643c-4aee-93f7-07ff90e853bb None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:57.952690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.953360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.953640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.953914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.954693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] RBAC: Adding query filter params (interface=public, service_id=5d782613825b4edfb4601ae054b96e56, region_id=RegionOne) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:57.955003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:57.956361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:57.961463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 507/1038] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/endpoints?service_id=5d782613825b4edfb4601ae054b96e56&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:57.962254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:57.980438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.982525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:57.983257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:57.983545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:57.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:57.984824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:57.984950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:57.986238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:57.988729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:57.999847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 532/1039] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:58.000694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.069166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e1fc9-515b-44f9-ab30-152f39de6247 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.104122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.104829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.105110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.105381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.106117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:58.106418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.107771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.108533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340767-946b-4b94-b5e0-399c13ebd7c7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.116141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 508/1040] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:58.116975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.146191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.146975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.147265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.147548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.148233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.150164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.157003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 533/1041] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:06:57 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:06:58.157803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.207937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:58.234259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da225-4a15-42d5-bb8b-edbc7b14b66a None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.236359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:58.276798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.277905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.278284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.278464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.285933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613afb-e67e-4215-84b8-afb0c21ebe7d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.287321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.288943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:06:58.304965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:58.306192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:06:58.307350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:58.308460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 509/1042] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:58.314200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.337984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.339430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.339430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.339430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] PATH_INFO: `/v3/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.359507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb1953b87ba4d1bb0522f1cb7a7cfd3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.360993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.362836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 534/1043] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3 => generated 285 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:58.368363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.427391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040f706-6906-4a98-8d69-833c9abb1c52 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.466819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.467506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.467785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.468054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.468929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:58.469236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.470664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.480988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe8cd4-9af3-4752-963c-c12fec0922d0 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.481458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:58.495136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 510/1044] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:58.502410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.506671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:58.529631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.530458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.530919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.531145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.613027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.615423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.629915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d02214-b1b2-4daf-a1fc-b889c89361f7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.671048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.672027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.672451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.672904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.734947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 535/1045] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:06:58.740527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.760252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:58.766527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:58.777363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.779827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.864402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e072-1d10-4191-8018-a8ff04f925ae None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:58.896738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 511/1046] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:06:58.904706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:58.923563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:58.924935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:58.925355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:58.925767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:58.927560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] RBAC: Authorizing `identity:get_project(project_id=4e236197fc4d437a89a5ff79addf075d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:58.930402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:58.938215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 536/1047] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:58.939747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:59.043599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:59.048032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:59.050945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54fac4-50ee-486e-8406-f534d30b894f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:59.071571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ef38e-75c1-4dbc-9ae4-823c211829ce None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:59.104727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:59.105890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:59.106353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:59.106824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:59.122885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:59.125418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:59.129778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:59.131098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:59.131815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:59.132524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:59.134824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=4e236197fc4d437a89a5ff79addf075d, user.id=4fb1953b87ba4d1bb0522f1cb7a7cfd3) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:59.135638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:59.138516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:59.148517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 537/1048] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/role_assignments?user.id=4fb1953b87ba4d1bb0522f1cb7a7cfd3&scope.project.id=4e236197fc4d437a89a5ff79addf075d&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:06:59.152865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:59.154624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:59.158338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:06:59.158338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:06:59.158338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:06:59.158338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:59.158338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 512/1049] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:06:59.162199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:59.286277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c728159-b776-44d6-97f6-ee3d2c8dbec6 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:59.330893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:59.331665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:59.332072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:59.332477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:59.333422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:59.333872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:59.335375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:59.348911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 538/1050] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:59.355168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:59.464934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9713370-af44-4351-935c-ed29bbb68415 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:59.503197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:59.504179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:59.504614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:59.505023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:59.512834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:59.514432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:06:59.528842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:06:59.530555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:06:59.533214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:06:59.536971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 513/1051] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:06:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:06:59.538823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:59.552161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:59.654389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8b7c2-07c7-42aa-8d9b-bc0908dab5c2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:59.693688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe712cb-6779-480f-adf9-9ef0d79099a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:59.694623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe712cb-6779-480f-adf9-9ef0d79099a1 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:59.695071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe712cb-6779-480f-adf9-9ef0d79099a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:59.695773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe712cb-6779-480f-adf9-9ef0d79099a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:59.696774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe712cb-6779-480f-adf9-9ef0d79099a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:59.697246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe712cb-6779-480f-adf9-9ef0d79099a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:59.698841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe712cb-6779-480f-adf9-9ef0d79099a1 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:59.717495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 539/1052] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:06:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:06:59.719092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:06:59.732300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:06:59.834515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12244098-13ff-4b21-b38d-9601301c1e79 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:06:59.873110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf005669-4a01-4a1e-8ddc-b180959a98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:06:59.874011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf005669-4a01-4a1e-8ddc-b180959a98b2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:06:59.874452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf005669-4a01-4a1e-8ddc-b180959a98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:06:59.874883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf005669-4a01-4a1e-8ddc-b180959a98b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:06:59.875878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf005669-4a01-4a1e-8ddc-b180959a98b2 None admin] RBAC: Adding query filter params (role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=48647bb993344a8b91cec7f623aea007) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:06:59.876370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf005669-4a01-4a1e-8ddc-b180959a98b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:06:59.878044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf005669-4a01-4a1e-8ddc-b180959a98b2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:06:59.892673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 514/1053] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:06:59 2022] GET /identity/v3/role_assignments?user.id=48647bb993344a8b91cec7f623aea007&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:00.250216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cc1e4-143f-4486-add7-f5157322f047 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:00.250737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cc1e4-143f-4486-add7-f5157322f047 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:00.250855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cc1e4-143f-4486-add7-f5157322f047 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:00.252719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 540/1054] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:00.259869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d456e89-f2be-402f-b6a4-35645168dbb0 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:00.260370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d456e89-f2be-402f-b6a4-35645168dbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:00.261034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d456e89-f2be-402f-b6a4-35645168dbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:00.289885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5d456e89-f2be-402f-b6a4-35645168dbb0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:00.291885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5d456e89-f2be-402f-b6a4-35645168dbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:00.409661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 515/1055] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:07:00.433883 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cc1e4-143f-4486-add7-f5157322f047 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:00.514482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:00.551381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cc1e4-143f-4486-add7-f5157322f047 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:00.590969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:00.591829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:00.592226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:00.592534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] PATH_INFO: `/v3/services/image` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:00.593278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:00.594655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:00.607496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:00.610105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:07:00.611823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:00.620202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 541/1056] 10.222.0.52 () {56 vars in 1105 bytes} [Fri Jun 24 09:07:00 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:00.620506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d456e89-f2be-402f-b6a4-35645168dbb0 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:00.620506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:00.786676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d456e89-f2be-402f-b6a4-35645168dbb0 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:00.843777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:00.844857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:00.845358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:00.845844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:00.847054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:00.847542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:00.850017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:00.863720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 516/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:00.866725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:01.032691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d40a-18c0-4971-aed5-cdc379265417 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:01.076575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b89342-13b9-410d-bb5a-56748635b27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:01.077274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b89342-13b9-410d-bb5a-56748635b27c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:01.077558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b89342-13b9-410d-bb5a-56748635b27c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:01.077840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b89342-13b9-410d-bb5a-56748635b27c None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:01.078624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b89342-13b9-410d-bb5a-56748635b27c None admin] RBAC: Adding query filter params (type=image) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:01.078937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b89342-13b9-410d-bb5a-56748635b27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:01.080310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b89342-13b9-410d-bb5a-56748635b27c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:01.082062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:01.092804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 542/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:01.094754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:01.205463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:01.265753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53399320-ac5d-4ae3-8c00-1b5cbc5973d3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:01.323075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3b252-17c7-42f9-90cf-3763851c8326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:01.324177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3b252-17c7-42f9-90cf-3763851c8326 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:01.324696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3b252-17c7-42f9-90cf-3763851c8326 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:01.325181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3b252-17c7-42f9-90cf-3763851c8326 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:01.326546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3b252-17c7-42f9-90cf-3763851c8326 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:01.329014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3b252-17c7-42f9-90cf-3763851c8326 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:01.360744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 517/1059] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:07:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:07:01.562944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c15c-8b7e-4fc3-b174-533750b53695 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:01.563455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c15c-8b7e-4fc3-b174-533750b53695 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:01.563571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c15c-8b7e-4fc3-b174-533750b53695 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:01.565505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 543/1060] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:07:01.571203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb139-6a65-4ce3-a224-d8b5e7945eec None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:01.571552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb139-6a65-4ce3-a224-d8b5e7945eec None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:01.571784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb139-6a65-4ce3-a224-d8b5e7945eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:01.589259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ef1eb139-6a65-4ce3-a224-d8b5e7945eec None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:01.590460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ef1eb139-6a65-4ce3-a224-d8b5e7945eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:01.674130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 518/1061] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:01.698215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1c15c-8b7e-4fc3-b174-533750b53695 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:01.874264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1c15c-8b7e-4fc3-b174-533750b53695 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:01.933296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5571a-19d5-4b8a-b71b-1723d8df8ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:01.934588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5571a-19d5-4b8a-b71b-1723d8df8ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:01.935077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5571a-19d5-4b8a-b71b-1723d8df8ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:01.935535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5571a-19d5-4b8a-b71b-1723d8df8ce6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:01.955284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5571a-19d5-4b8a-b71b-1723d8df8ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2aa5571a-19d5-4b8a-b71b-1723d8df8ce6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:01.970813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:01.971796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:01.973083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:01.973735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 544/1062] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:01.981123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1eb139-6a65-4ce3-a224-d8b5e7945eec None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:01.992889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f2eb4-2a94-4560-8e0d-78dab79ec7a3 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:01.993180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f2eb4-2a94-4560-8e0d-78dab79ec7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:01.993448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f2eb4-2a94-4560-8e0d-78dab79ec7a3 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:01.994819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 545/1063] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:07:02.000044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b59ac-714f-4eeb-8dcb-d434b74484a5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.000441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b59ac-714f-4eeb-8dcb-d434b74484a5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.000648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b59ac-714f-4eeb-8dcb-d434b74484a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.018793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a39b59ac-714f-4eeb-8dcb-d434b74484a5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:02.019994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a39b59ac-714f-4eeb-8dcb-d434b74484a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:02.060358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:02.099545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:02.118355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 546/1064] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:07:02.135001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1eb139-6a65-4ce3-a224-d8b5e7945eec None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.145546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b59ac-714f-4eeb-8dcb-d434b74484a5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:02.176118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.176876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.177152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.177560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.178604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:02.178907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:02.180307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:02.198939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 519/1065] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:02.201471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:02.280305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b59ac-714f-4eeb-8dcb-d434b74484a5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.317174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.318001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.318283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.318560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.332375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:02.334666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edeaa5-8c99-4c8b-94e1-ae90b95ab2f3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:02.346020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:02.347702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:07:02.349394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:02.351721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:02.355873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 547/1066] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Jun 24 09:07:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:02.356911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:02.383795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.385218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.385693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.386162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] PATH_INFO: `/v3/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.388709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb1953b87ba4d1bb0522f1cb7a7cfd3)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:02.391174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:02.394040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 520/1067] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:02 2022] GET /identity/v3/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3 => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:02.401149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:02.437516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:02.500230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815ccfc-38be-4276-8286-9041adff901c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.525866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c871a-c062-4386-bb34-489eb6a10152 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.535946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.536646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.536927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.537196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.538231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:02.538231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:02.539616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:02.549118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 548/1068] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Jun 24 09:07:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:02.554525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:02.568283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.569202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.569483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.569756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.571507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] RBAC: Authorizing `identity:get_project(project_id=4e236197fc4d437a89a5ff79addf075d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:02.572981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:02.579651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 521/1069] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:02 2022] GET /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:02.580765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:02.629432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:02.665121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089f8f9-e012-45e3-863f-5c689f5453ec None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.675565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:02.693457 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9228b9-0432-4659-8e1b-98a0151981f5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.703181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.704215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.704978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.705343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.716611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:02.718035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:02.730137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.731027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.731312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.731588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.733581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] RBAC: Authorizing `identity:create_grant(project_id=4e236197fc4d437a89a5ff79addf075d, user_id=4fb1953b87ba4d1bb0522f1cb7a7cfd3, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:02.735006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:07:02.744670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:02.745597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:02.746519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:07:02.746519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:07:02.746519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:02.746519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:02.746742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 549/1070] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:02.753079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:02.765046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 522/1071] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:07:02 2022] PUT /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:07:02.918389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fdec3-3736-4f08-98fc-981a202dfd52 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:02.954897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:02.955561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:02.955908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:02.956188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:02.956931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:02.957239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:02.958595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:02.970865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 550/1072] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:02.976272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:03.086203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886231c-fc83-40c5-b3f0-2678365d698f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:03.093462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:03.124157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:03.124981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.125263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.125538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:03.133527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:03.134972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:07:03.149267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:03.150286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:07:03.151232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:03.152621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 523/1073] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:07:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:03.156252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:03.183734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:03.263983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507689a8-275b-4db1-b9ac-b1b125434513 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:03.300153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:03.300762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.301047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.301321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:03.302156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:03.302467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:03.303882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:03.327351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 551/1074] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:07:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:03.328724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:03.432273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a4aa-c9b7-4e83-be5e-3b7e664e1bb7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:03.466571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21705549-8674-4b02-a866-9890842217a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:03.467606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21705549-8674-4b02-a866-9890842217a8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.467896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21705549-8674-4b02-a866-9890842217a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.468172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21705549-8674-4b02-a866-9890842217a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:03.479185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21705549-8674-4b02-a866-9890842217a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:03.480658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21705549-8674-4b02-a866-9890842217a8 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-21705549-8674-4b02-a866-9890842217a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:07:03.498370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:03.499376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:03.500263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:07:03.500263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:07:03.500263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:03.500263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:03.501071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 524/1075] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:03.505378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:03.590544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:03.611013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea530e32-e1db-4d75-988c-cd2608100319 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:03.647022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0899f2f-54c5-4cf5-8cb2-c56c76555149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:03.647687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899f2f-54c5-4cf5-8cb2-c56c76555149 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.647972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899f2f-54c5-4cf5-8cb2-c56c76555149 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.648276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899f2f-54c5-4cf5-8cb2-c56c76555149 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:03.649023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0899f2f-54c5-4cf5-8cb2-c56c76555149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:03.649330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0899f2f-54c5-4cf5-8cb2-c56c76555149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:03.650705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0899f2f-54c5-4cf5-8cb2-c56c76555149 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:03.667655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 552/1076] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:03.669000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21705549-8674-4b02-a866-9890842217a8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:03.690229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:03.699891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad829e-250d-4db1-88b2-69f187dea252 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.700178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad829e-250d-4db1-88b2-69f187dea252 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.700456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad829e-250d-4db1-88b2-69f187dea252 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:03.701981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 553/1077] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:07:03.706763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83807ece-3ac4-4720-b427-54cbb749e719 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.707055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83807ece-3ac4-4720-b427-54cbb749e719 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.707336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83807ece-3ac4-4720-b427-54cbb749e719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:03.727871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-83807ece-3ac4-4720-b427-54cbb749e719 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:03.729144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-83807ece-3ac4-4720-b427-54cbb749e719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:03.781279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21705549-8674-4b02-a866-9890842217a8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:03.807703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 554/1078] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:07:03.823184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:03.823988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.824266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.824539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:03.829694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83807ece-3ac4-4720-b427-54cbb749e719 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:03.831984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:03.833467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:07:03.847897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:03.848850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:03.850346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 525/1079] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:07:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:03.855254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:03.910430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:03.941310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:03.946312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83807ece-3ac4-4720-b427-54cbb749e719 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:03.963570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1b994-bbc1-4536-9745-65fa8d4829a1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:03.983844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:03.984492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:03.984807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:03.985077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:04.001316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:04.002005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:04.002261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:04.002529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:04.003311 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:04.003600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:04.005039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:04.021704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 526/1080] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:07:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:04.022844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:04.065078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:04.066526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:04.134561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994f058-1248-474a-a7ec-f7e6221b53e9 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:04.138411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:04.175761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 555/1081] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:04.176033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:04.176878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:04.177255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:04.177624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/48647bb993344a8b91cec7f623aea007/roles/54ceb2d228bc4fefb2dd6fcbc299a1ea` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:04.180018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=48647bb993344a8b91cec7f623aea007, role_id=54ceb2d228bc4fefb2dd6fcbc299a1ea)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:04.181485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:04.181668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:04.192276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:04.199321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 527/1082] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:07:04 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/48647bb993344a8b91cec7f623aea007/roles/54ceb2d228bc4fefb2dd6fcbc299a1ea => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:07:04.318607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c51f-68bb-4ce5-9aeb-96c81caa529b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:04.364481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759fb78-1f38-4940-bdd1-0c3b8da5d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:04.365323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759fb78-1f38-4940-bdd1-0c3b8da5d422 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:04.365602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759fb78-1f38-4940-bdd1-0c3b8da5d422 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:04.365873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759fb78-1f38-4940-bdd1-0c3b8da5d422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:04.379740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759fb78-1f38-4940-bdd1-0c3b8da5d422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:04.381266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759fb78-1f38-4940-bdd1-0c3b8da5d422 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8759fb78-1f38-4940-bdd1-0c3b8da5d422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:07:04.405176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:04.406113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:04.407010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:07:04.407010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:07:04.407010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:04.407010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:04.408999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 556/1083] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:04.415412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:04.527483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c69c-3419-47bb-833f-cd30ac891e25 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:04.565894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f78af8-d944-4c3e-915f-0272df986da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:04.566671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f78af8-d944-4c3e-915f-0272df986da1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:04.566960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f78af8-d944-4c3e-915f-0272df986da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:04.567229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f78af8-d944-4c3e-915f-0272df986da1 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:04.567952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f78af8-d944-4c3e-915f-0272df986da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:04.568257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f78af8-d944-4c3e-915f-0272df986da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:04.569784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f78af8-d944-4c3e-915f-0272df986da1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:04.584669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 528/1084] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:05.299451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce229d6-ad8f-47b3-bce6-e7a4e4ad5b65 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:05.300026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce229d6-ad8f-47b3-bce6-e7a4e4ad5b65 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:05.300026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce229d6-ad8f-47b3-bce6-e7a4e4ad5b65 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:05.301336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 557/1085] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:05.307326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b090-2d36-45e7-a887-7da3c56edd9c None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:05.307610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b090-2d36-45e7-a887-7da3c56edd9c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:05.307886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b090-2d36-45e7-a887-7da3c56edd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:05.324366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-37b6b090-2d36-45e7-a887-7da3c56edd9c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:05.325575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-37b6b090-2d36-45e7-a887-7da3c56edd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:05.379875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:05.394189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 529/1086] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:05.417536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce229d6-ad8f-47b3-bce6-e7a4e4ad5b65 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:05.448320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:05.529072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce229d6-ad8f-47b3-bce6-e7a4e4ad5b65 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:05.564052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:05.564799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:05.565088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:05.565367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:05.575829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:05.583457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:05.584529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:05.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:05.588718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 558/1087] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:05.591094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b090-2d36-45e7-a887-7da3c56edd9c None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:05.688120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b090-2d36-45e7-a887-7da3c56edd9c None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:05.721732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:05.722402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:05.722686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:05.722959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:05.723711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:05.724017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:05.725406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:05.735746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 530/1088] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:05.741640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:05.823491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:05.842353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706559e-2caa-410c-898e-32b48d6dafdd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:05.875380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:05.876259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:05.876542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:05.876846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] PATH_INFO: `/v3/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:05.878031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] RBAC: Authorizing `identity:get_user(user_id=4fb1953b87ba4d1bb0522f1cb7a7cfd3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:05.879459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:05.881122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 559/1089] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:05 2022] GET /identity/v3/users/4fb1953b87ba4d1bb0522f1cb7a7cfd3 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:05.886952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:05.936877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:05.985970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d238c-1cd7-46bd-a378-3676ced4d447 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:06.019425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:06.020084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:06.020364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:06.020670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:06.088591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:06.089947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:06.136952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:06.187343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 531/1090] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:06.193205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:06.292878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41ef90-2b24-42d8-8e67-6d4a8bb1a45e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:06.327001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac91cb6-e26e-46e1-9378-1ce12981a108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:06.327860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac91cb6-e26e-46e1-9378-1ce12981a108 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:06.328136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac91cb6-e26e-46e1-9378-1ce12981a108 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:06.328406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac91cb6-e26e-46e1-9378-1ce12981a108 None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:06.329615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac91cb6-e26e-46e1-9378-1ce12981a108 None admin] RBAC: Authorizing `identity:get_project(project_id=4e236197fc4d437a89a5ff79addf075d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:06.331045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac91cb6-e26e-46e1-9378-1ce12981a108 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:06.332791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 560/1091] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:06 2022] GET /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:06.339501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:06.438323 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7def7e-c03e-4b2c-8cd1-994fa2e060d1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:06.472805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d86df-1fa2-481f-8013-808f82bb2715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:06.473463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d86df-1fa2-481f-8013-808f82bb2715 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:06.473749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d86df-1fa2-481f-8013-808f82bb2715 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:06.474029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d86df-1fa2-481f-8013-808f82bb2715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:06.474859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d86df-1fa2-481f-8013-808f82bb2715 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=4e236197fc4d437a89a5ff79addf075d, user.id=4fb1953b87ba4d1bb0522f1cb7a7cfd3) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:06.475172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d86df-1fa2-481f-8013-808f82bb2715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:06.476657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d86df-1fa2-481f-8013-808f82bb2715 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:06.482569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 532/1092] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:06 2022] GET /identity/v3/role_assignments?user.id=4fb1953b87ba4d1bb0522f1cb7a7cfd3&scope.project.id=4e236197fc4d437a89a5ff79addf075d&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:06.797248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b49809-6f88-42ab-9abe-7e3619d4c17f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:06.797664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b49809-6f88-42ab-9abe-7e3619d4c17f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:06.798052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b49809-6f88-42ab-9abe-7e3619d4c17f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:06.799603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 561/1093] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:06.805193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd73a32-8c19-41e4-a6a2-e232273c3df6 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:06.806331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd73a32-8c19-41e4-a6a2-e232273c3df6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:06.806725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd73a32-8c19-41e4-a6a2-e232273c3df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:06.824451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-fcd73a32-8c19-41e4-a6a2-e232273c3df6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:06.825653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-fcd73a32-8c19-41e4-a6a2-e232273c3df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:06.835993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:06.895627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 533/1094] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:06.919844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b49809-6f88-42ab-9abe-7e3619d4c17f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:06.941160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:07.031586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b49809-6f88-42ab-9abe-7e3619d4c17f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:07.074351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:07.075083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:07.075370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:07.075650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:07.087387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:07.095588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:07.097553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:07:07.099460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:07.102030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 562/1095] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Jun 24 09:07:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:07.103140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd73a32-8c19-41e4-a6a2-e232273c3df6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:07.203437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd73a32-8c19-41e4-a6a2-e232273c3df6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:07.239967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b6473-3699-4349-949e-046055d43550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:07.240667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b6473-3699-4349-949e-046055d43550 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:07.240948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b6473-3699-4349-949e-046055d43550 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:07.241231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b6473-3699-4349-949e-046055d43550 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:07.242515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b6473-3699-4349-949e-046055d43550 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:07.242820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b6473-3699-4349-949e-046055d43550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:07.244182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b6473-3699-4349-949e-046055d43550 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:07.252855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 534/1096] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Jun 24 09:07:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:07.259102 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:07.340629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:07.362717 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8bed7-6ae9-4674-9fd3-815a9a223cb7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:07.397019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:07.397804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:07.398082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:07.398489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:07.408400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:07.409859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:07:07.426732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:07.434857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:07.436195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:07:07.436195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:07:07.436195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:07.436195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:07.436195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 563/1097] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:07.436645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b6473-3699-4349-949e-046055d43550 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:07.446090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:07.536128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b6473-3699-4349-949e-046055d43550 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:07.573240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:07.573900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:07.574176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:07.574449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:07.575158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:07.575459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:07.576849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:07.588842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 535/1098] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:07.594443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:07.695709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a1356-6dc1-459e-9aac-ec878b8fcaf8 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:07.724213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:07.732767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:07.733562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:07.733845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:07.734115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:07.741370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:07.743412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:07:07.757703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:07.759399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:07.760438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 564/1099] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Jun 24 09:07:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:07.764694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:07.830194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:07.865331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ba5f3-fd98-4204-afcf-e7e4c0d70944 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:07.898694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:07.899373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:07.899656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:07.899934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:07.900815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:07.901130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:07.902553 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:07.920284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 536/1100] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Jun 24 09:07:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:07.926164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.030277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c2ee6-1b88-4adb-a5d2-86fd7350f198 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:08.066789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:08.067447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.067733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.068019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.126429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:08.137141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:08.138594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:08.238920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 565/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:07:08.244648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.344092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6c34f-60c5-4ded-b2bf-d02e14c51eab None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:08.367288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:08.378664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7f0ac-cfad-4952-9358-43ad6cf3d0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:08.379390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7f0ac-cfad-4952-9358-43ad6cf3d0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.379673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7f0ac-cfad-4952-9358-43ad6cf3d0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.379949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7f0ac-cfad-4952-9358-43ad6cf3d0a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.392375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7f0ac-cfad-4952-9358-43ad6cf3d0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:08.393880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7f0ac-cfad-4952-9358-43ad6cf3d0a3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-14a7f0ac-cfad-4952-9358-43ad6cf3d0a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:07:08.413273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:08.414265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:08.415375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:07:08.415375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:07:08.415375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:07:08.415375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:08.415375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 537/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:08.420393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.513760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:08.524750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e1c1-0b96-4fd9-9671-0ed84ea89a82 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:08.567185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13923-b335-4709-ad7a-1cb79bc9076c None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.567472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13923-b335-4709-ad7a-1cb79bc9076c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.567592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:08.567756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13923-b335-4709-ad7a-1cb79bc9076c None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.568186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.568478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.568789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.569168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 538/1103] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:07:08.569591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:08.569900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:08.571908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:08.574969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b9f6-31cb-4ab7-903c-9ca4b39b0bc5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.575257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b9f6-31cb-4ab7-903c-9ca4b39b0bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.575539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b9f6-31cb-4ab7-903c-9ca4b39b0bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.590618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 566/1104] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:08.591709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.592915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-2f52b9f6-31cb-4ab7-903c-9ca4b39b0bc5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:08.594106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2f52b9f6-31cb-4ab7-903c-9ca4b39b0bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:08.663402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 539/1105] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:07:08.689223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b9f6-31cb-4ab7-903c-9ca4b39b0bc5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.693965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a5040-31da-4268-8d3d-8a590029ec5a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:08.728502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:08.729332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.729647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.729926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.737346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:08.738929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:07:08.754006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:08.755129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:08.756055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 567/1106] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:08.761681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.774199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:08.795861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b9f6-31cb-4ab7-903c-9ca4b39b0bc5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:08.804993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:08.833111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:08.833912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.834189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.834461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.835669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:08.837154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:08.839015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 540/1107] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:08.844530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.864859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daec25-7950-4ad5-9c70-09239b3f2490 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:08.899488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:08.900143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.900429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.900731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.901506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:08.901808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:08.903212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:08.915475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 568/1108] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:07:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:08.921519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:08.947254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f24df-586b-456d-ba1c-7083b9061df2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:08.981993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0332e5e-9d08-409b-81c8-d8bb25e6b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:08.982685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0332e5e-9d08-409b-81c8-d8bb25e6b46f None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:08.982955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0332e5e-9d08-409b-81c8-d8bb25e6b46f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:08.983233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0332e5e-9d08-409b-81c8-d8bb25e6b46f None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:08.983944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0332e5e-9d08-409b-81c8-d8bb25e6b46f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:08.985433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0332e5e-9d08-409b-81c8-d8bb25e6b46f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:08.999964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:09.007716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 541/1109] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:07:08 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:07:09.034860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b71de-6dac-4142-a39f-4ea7ccabd562 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:09.076207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ea5cc-fc40-495b-a4d3-3e8033d23186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:09.077023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ea5cc-fc40-495b-a4d3-3e8033d23186 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:09.077294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ea5cc-fc40-495b-a4d3-3e8033d23186 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:09.077572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ea5cc-fc40-495b-a4d3-3e8033d23186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:09.078471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ea5cc-fc40-495b-a4d3-3e8033d23186 None admin] RBAC: Adding query filter params (role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=48647bb993344a8b91cec7f623aea007) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:09.078783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ea5cc-fc40-495b-a4d3-3e8033d23186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:09.080232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ea5cc-fc40-495b-a4d3-3e8033d23186 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:09.087195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 569/1110] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:08 2022] GET /identity/v3/role_assignments?user.id=48647bb993344a8b91cec7f623aea007&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=54ceb2d228bc4fefb2dd6fcbc299a1ea => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:11.178217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe753f-d5ba-4dcd-866d-68f30c685f15 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:11.178763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe753f-d5ba-4dcd-866d-68f30c685f15 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:11.178763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe753f-d5ba-4dcd-866d-68f30c685f15 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:11.180155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 542/1111] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:11.185838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581df12-c0ac-4169-a022-6909fe2cccab None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:11.186306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581df12-c0ac-4169-a022-6909fe2cccab None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:11.186552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581df12-c0ac-4169-a022-6909fe2cccab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:11.204729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d581df12-c0ac-4169-a022-6909fe2cccab None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:11.205899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d581df12-c0ac-4169-a022-6909fe2cccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:11.278271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 570/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:11.303373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe753f-d5ba-4dcd-866d-68f30c685f15 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:11.336932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:11.414889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe753f-d5ba-4dcd-866d-68f30c685f15 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:11.450034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:11.450882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:11.451159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:11.451431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] PATH_INFO: `/v3/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:11.452923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] RBAC: Authorizing `identity:get_role(role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:11.454282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:11.455705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 543/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:11 2022] GET /identity/v3/roles/cc730cc298404131a99c3547377a5e49 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:11.462944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581df12-c0ac-4169-a022-6909fe2cccab None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:11.526302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:11.564943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581df12-c0ac-4169-a022-6909fe2cccab None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:11.599809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:11.600698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:11.600985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:11.601281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] PATH_INFO: `/v3/users/eb362b3a86684b968abcab395802d7fc` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:11.616131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] RBAC: Authorizing `identity:get_user(user_id=eb362b3a86684b968abcab395802d7fc)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:11.618139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:11.619797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 571/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:11 2022] GET /identity/v3/users/eb362b3a86684b968abcab395802d7fc => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:11.625680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:11.733230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:11.737186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19124d-c974-432e-bfe2-78136a8c96a9 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:11.782341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:11.782991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:11.783274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:11.783550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:11.855491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:11.856888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:11.951435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:11.956121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 544/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:11.961782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:12.071694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d7b61-33c2-4927-8830-6b327dc5aed4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:12.093729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:12.108300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d886f2-7fc7-407f-800b-420a2b91f4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:12.109120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d886f2-7fc7-407f-800b-420a2b91f4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:12.109398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d886f2-7fc7-407f-800b-420a2b91f4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:12.109671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d886f2-7fc7-407f-800b-420a2b91f4e3 None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:12.110839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d886f2-7fc7-407f-800b-420a2b91f4e3 None admin] RBAC: Authorizing `identity:get_project(project_id=4e236197fc4d437a89a5ff79addf075d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:12.112248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d886f2-7fc7-407f-800b-420a2b91f4e3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:12.114234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 572/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:11 2022] GET /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:12.120926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:12.229499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda5107-c6ac-4714-b2b4-78ea74018496 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:12.263401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66b7f-a5a6-4500-95ed-7d2eb129b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:12.264044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66b7f-a5a6-4500-95ed-7d2eb129b299 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:12.264321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66b7f-a5a6-4500-95ed-7d2eb129b299 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:12.264617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66b7f-a5a6-4500-95ed-7d2eb129b299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:12.265439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d66b7f-a5a6-4500-95ed-7d2eb129b299 None admin] RBAC: Adding query filter params (role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=4e236197fc4d437a89a5ff79addf075d, user.id=eb362b3a86684b968abcab395802d7fc) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:12.265744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d66b7f-a5a6-4500-95ed-7d2eb129b299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:12.267165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d66b7f-a5a6-4500-95ed-7d2eb129b299 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:12.272784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 545/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:12 2022] GET /identity/v3/role_assignments?user.id=eb362b3a86684b968abcab395802d7fc&scope.project.id=4e236197fc4d437a89a5ff79addf075d&role.id=cc730cc298404131a99c3547377a5e49 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:14.319777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ca70-0d0d-4097-926e-42e922aa4e8f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:14.320846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ca70-0d0d-4097-926e-42e922aa4e8f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:14.320846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ca70-0d0d-4097-926e-42e922aa4e8f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:14.322290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 573/1118] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:14.329222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7e7a-8d2e-43af-888b-fd68b0997dfb None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:14.329617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7e7a-8d2e-43af-888b-fd68b0997dfb None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:14.329819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7e7a-8d2e-43af-888b-fd68b0997dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:14.347681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-793d7e7a-8d2e-43af-888b-fd68b0997dfb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:14.348907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-793d7e7a-8d2e-43af-888b-fd68b0997dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:14.408754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:14.421706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 546/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:14.446726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1ca70-0d0d-4097-926e-42e922aa4e8f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:14.550102 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1ca70-0d0d-4097-926e-42e922aa4e8f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:14.585156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:14.586509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:14.586789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:14.587061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] PATH_INFO: `/v3/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:14.588585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] RBAC: Authorizing `identity:get_role(role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:14.589962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:14.591351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 574/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:14 2022] GET /identity/v3/roles/cc730cc298404131a99c3547377a5e49 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:14.597304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7e7a-8d2e-43af-888b-fd68b0997dfb None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:14.697881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7e7a-8d2e-43af-888b-fd68b0997dfb None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:14.731673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:14.732548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:14.732859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:14.733134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] PATH_INFO: `/v3/users/eb362b3a86684b968abcab395802d7fc` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:14.734325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] RBAC: Authorizing `identity:get_user(user_id=eb362b3a86684b968abcab395802d7fc)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:14.735768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:14.737438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 547/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:14 2022] GET /identity/v3/users/eb362b3a86684b968abcab395802d7fc => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:14.743904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:14.808656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:14.856327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31df931-c71a-481e-b6ef-7521d77fe810 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:14.890492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bba01f-7296-4931-aae0-a85a18cb45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:14.891343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba01f-7296-4931-aae0-a85a18cb45f0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:14.891626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba01f-7296-4931-aae0-a85a18cb45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:14.891902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba01f-7296-4931-aae0-a85a18cb45f0 None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:14.893094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bba01f-7296-4931-aae0-a85a18cb45f0 None admin] RBAC: Authorizing `identity:get_project(project_id=4e236197fc4d437a89a5ff79addf075d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:14.894526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bba01f-7296-4931-aae0-a85a18cb45f0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:14.896201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 575/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:14 2022] GET /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:14.902191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:14.959778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:15.000481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f03f86-21be-4fd8-bb81-7e3df0c098c1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:15.032724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd443-e8ba-4d2b-b2ff-7c3d76c951d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:15.033611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd443-e8ba-4d2b-b2ff-7c3d76c951d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:15.033894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd443-e8ba-4d2b-b2ff-7c3d76c951d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:15.034171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd443-e8ba-4d2b-b2ff-7c3d76c951d2 None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d/users/eb362b3a86684b968abcab395802d7fc/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:15.036340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd443-e8ba-4d2b-b2ff-7c3d76c951d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e236197fc4d437a89a5ff79addf075d, user_id=eb362b3a86684b968abcab395802d7fc, role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:15.037783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd443-e8ba-4d2b-b2ff-7c3d76c951d2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:15.050588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 548/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:07:14 2022] PUT /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d/users/eb362b3a86684b968abcab395802d7fc/roles/cc730cc298404131a99c3547377a5e49 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:07:17.045811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ccc2-6215-476c-96a6-24b2f162d94d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:17.046215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ccc2-6215-476c-96a6-24b2f162d94d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:17.046370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ccc2-6215-476c-96a6-24b2f162d94d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:17.047728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 576/1124] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:17.053026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2384891-bd38-457f-afcb-cec71ea12d49 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:17.053469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2384891-bd38-457f-afcb-cec71ea12d49 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:17.053572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2384891-bd38-457f-afcb-cec71ea12d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:17.070166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f2384891-bd38-457f-afcb-cec71ea12d49 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:17.071355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f2384891-bd38-457f-afcb-cec71ea12d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:17.140067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 549/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:17.164758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34ccc2-6215-476c-96a6-24b2f162d94d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:17.263537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34ccc2-6215-476c-96a6-24b2f162d94d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:17.288937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:17.298152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:17.299003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:17.299291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:17.299569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] PATH_INFO: `/v3/roles/cc730cc298404131a99c3547377a5e49` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:17.301077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] RBAC: Authorizing `identity:get_role(role_id=cc730cc298404131a99c3547377a5e49)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:17.302444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:17.303806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 577/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:17 2022] GET /identity/v3/roles/cc730cc298404131a99c3547377a5e49 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:17.309950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2384891-bd38-457f-afcb-cec71ea12d49 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:17.358600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:17.407936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2384891-bd38-457f-afcb-cec71ea12d49 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:17.440419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:17.441305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:17.441590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:17.441868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] PATH_INFO: `/v3/users/eb362b3a86684b968abcab395802d7fc` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:17.443036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] RBAC: Authorizing `identity:get_user(user_id=eb362b3a86684b968abcab395802d7fc)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:17.444463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:17.446645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 550/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:17 2022] GET /identity/v3/users/eb362b3a86684b968abcab395802d7fc => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:17.452413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:17.550529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd94e2-9bb4-4b86-b9ac-98452f553b72 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:17.583549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:17.584188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:17.584475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:17.584781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:17.643219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:17.650732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:17.652171 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:17.747558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 578/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:17.753442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:17.849008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db9e6-6584-4d4f-b7d1-40d4c78e05cb None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:17.865264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:17.882886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a132f-7bac-4521-8bfb-deb55c0dc826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:17.883775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a132f-7bac-4521-8bfb-deb55c0dc826 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:17.884056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a132f-7bac-4521-8bfb-deb55c0dc826 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:17.884338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a132f-7bac-4521-8bfb-deb55c0dc826 None admin] PATH_INFO: `/v3/projects/4e236197fc4d437a89a5ff79addf075d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:17.885569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a132f-7bac-4521-8bfb-deb55c0dc826 None admin] RBAC: Authorizing `identity:get_project(project_id=4e236197fc4d437a89a5ff79addf075d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:17.887006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a132f-7bac-4521-8bfb-deb55c0dc826 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:17.888722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 551/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:17 2022] GET /identity/v3/projects/4e236197fc4d437a89a5ff79addf075d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:17.897306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:17.992414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:17.998991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be0c3e-8cb1-43f3-89c8-728ffc805179 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:18.031751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d671ef-fb9f-45ff-b60e-ba3d976b48b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:18.032419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d671ef-fb9f-45ff-b60e-ba3d976b48b3 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:18.032750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d671ef-fb9f-45ff-b60e-ba3d976b48b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:18.033026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d671ef-fb9f-45ff-b60e-ba3d976b48b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:18.033854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d671ef-fb9f-45ff-b60e-ba3d976b48b3 None admin] RBAC: Adding query filter params (role.id=cc730cc298404131a99c3547377a5e49, scope.project.id=4e236197fc4d437a89a5ff79addf075d, user.id=eb362b3a86684b968abcab395802d7fc) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:18.034164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d671ef-fb9f-45ff-b60e-ba3d976b48b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:18.035586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d671ef-fb9f-45ff-b60e-ba3d976b48b3 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:18.041962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 579/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:17 2022] GET /identity/v3/role_assignments?user.id=eb362b3a86684b968abcab395802d7fc&scope.project.id=4e236197fc4d437a89a5ff79addf075d&role.id=cc730cc298404131a99c3547377a5e49 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:20.073107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146960b4-46c8-46a2-8498-0148f1d8caa6 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:20.073625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146960b4-46c8-46a2-8498-0148f1d8caa6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:20.073719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146960b4-46c8-46a2-8498-0148f1d8caa6 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:20.075037 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 552/1131] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:20.080212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc3bd6-a8f4-40f6-ac7b-5dd3c5e4721c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:20.080641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc3bd6-a8f4-40f6-ac7b-5dd3c5e4721c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:20.080810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc3bd6-a8f4-40f6-ac7b-5dd3c5e4721c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:20.097716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ccbc3bd6-a8f4-40f6-ac7b-5dd3c5e4721c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:20.098755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ccbc3bd6-a8f4-40f6-ac7b-5dd3c5e4721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:20.167292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 580/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:20.190687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146960b4-46c8-46a2-8498-0148f1d8caa6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:20.290185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146960b4-46c8-46a2-8498-0148f1d8caa6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:20.324108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e0308-48df-43cc-83a8-9629780eba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:20.324921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0308-48df-43cc-83a8-9629780eba1b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:20.325200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0308-48df-43cc-83a8-9629780eba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:20.325476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0308-48df-43cc-83a8-9629780eba1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:20.326662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e0308-48df-43cc-83a8-9629780eba1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:20.328085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e0308-48df-43cc-83a8-9629780eba1b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:20.329720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 553/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:20.336222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc3bd6-a8f4-40f6-ac7b-5dd3c5e4721c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:20.419678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:20.432144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc3bd6-a8f4-40f6-ac7b-5dd3c5e4721c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:20.465639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e00b7-c2e1-4c2d-96bc-de34a50b4e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:20.466291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e00b7-c2e1-4c2d-96bc-de34a50b4e03 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:20.466577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e00b7-c2e1-4c2d-96bc-de34a50b4e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:20.466848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e00b7-c2e1-4c2d-96bc-de34a50b4e03 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:20.467553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e00b7-c2e1-4c2d-96bc-de34a50b4e03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:20.469003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e00b7-c2e1-4c2d-96bc-de34a50b4e03 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:20.492849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 581/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:07:20 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:07:22.655166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7576f55-1e60-449e-a9cc-a49eec7e3261 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:22.655695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7576f55-1e60-449e-a9cc-a49eec7e3261 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:22.655695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7576f55-1e60-449e-a9cc-a49eec7e3261 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:22.657108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 554/1135] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:22.662337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce009815-5322-43c2-9db4-a4fe39bc59b5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:22.662744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce009815-5322-43c2-9db4-a4fe39bc59b5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:22.662902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce009815-5322-43c2-9db4-a4fe39bc59b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:22.679308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ce009815-5322-43c2-9db4-a4fe39bc59b5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:22.680475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ce009815-5322-43c2-9db4-a4fe39bc59b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:22.746692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 582/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:07:22.772204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7576f55-1e60-449e-a9cc-a49eec7e3261 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:22.827462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:22.872012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7576f55-1e60-449e-a9cc-a49eec7e3261 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:22.904306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde23f6f-0497-4107-baa7-fc1d94c5bc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:22.905107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23f6f-0497-4107-baa7-fc1d94c5bc3d None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:22.905439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23f6f-0497-4107-baa7-fc1d94c5bc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:22.905662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde23f6f-0497-4107-baa7-fc1d94c5bc3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:22.906817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde23f6f-0497-4107-baa7-fc1d94c5bc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:22.908226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde23f6f-0497-4107-baa7-fc1d94c5bc3d None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:22.909854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 555/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:07:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:22.915845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce009815-5322-43c2-9db4-a4fe39bc59b5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:22.996113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:23.027379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce009815-5322-43c2-9db4-a4fe39bc59b5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:23.063321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67c6da-2946-4f59-ab36-e56f96f86484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:23.063983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67c6da-2946-4f59-ab36-e56f96f86484 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:23.064266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67c6da-2946-4f59-ab36-e56f96f86484 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:23.064539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67c6da-2946-4f59-ab36-e56f96f86484 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:23.065270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67c6da-2946-4f59-ab36-e56f96f86484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:23.066687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67c6da-2946-4f59-ab36-e56f96f86484 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:23.087044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 583/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:07:22 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:07:25.140365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ba0c-c64e-4060-a1fb-2375754be35f None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:25.140935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ba0c-c64e-4060-a1fb-2375754be35f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:25.140935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ba0c-c64e-4060-a1fb-2375754be35f None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:25.142368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 556/1139] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:07:25.148337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fc980-e6c6-4661-ac04-b71fc89dd989 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:25.148788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fc980-e6c6-4661-ac04-b71fc89dd989 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:25.148929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fc980-e6c6-4661-ac04-b71fc89dd989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:25.167136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2c8fc980-e6c6-4661-ac04-b71fc89dd989 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:25.168549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2c8fc980-e6c6-4661-ac04-b71fc89dd989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:25.237016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 584/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:25.260806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ba0c-c64e-4060-a1fb-2375754be35f None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:25.363820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ba0c-c64e-4060-a1fb-2375754be35f None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:25.387824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:25.400949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:25.401746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:25.402026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:25.402296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:25.412843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:25.420997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:25.422085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:25.422957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 557/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:25.428188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fc980-e6c6-4661-ac04-b71fc89dd989 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:25.486244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:25.530598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fc980-e6c6-4661-ac04-b71fc89dd989 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:25.562791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:25.563447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:25.563726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:25.563998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:25.564759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:25.565064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:25.566416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:25.574653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 585/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:25.580468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:25.675283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb45ab-5518-4a47-9625-7be7e9073b82 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:25.707152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:25.708066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:25.708345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:25.708649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] PATH_INFO: `/v3/users/17265851b6ba4f088414a36bec764346` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:25.721852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] RBAC: Authorizing `identity:get_user(user_id=17265851b6ba4f088414a36bec764346)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:25.723268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:25.724918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 558/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:25 2022] GET /identity/v3/users/17265851b6ba4f088414a36bec764346 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:25.730571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:25.829108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc30e0-5a20-45a2-ae93-53a8201cdfac None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:25.850099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:25.867632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:25.868276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:25.868578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:25.868858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:25.933564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:25.934930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:26.028858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 586/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:26.034687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:26.076288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:26.131641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6e689-bedd-42dc-8648-5ee43250d3d3 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:26.167447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570440ed-f973-4c51-969f-3b0cd5cc5a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:26.168350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570440ed-f973-4c51-969f-3b0cd5cc5a33 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:26.168675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570440ed-f973-4c51-969f-3b0cd5cc5a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:26.168957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570440ed-f973-4c51-969f-3b0cd5cc5a33 None admin] PATH_INFO: `/v3/projects/097213bd8ea048a6b079a72065d5f1f4` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:26.170151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570440ed-f973-4c51-969f-3b0cd5cc5a33 None admin] RBAC: Authorizing `identity:get_project(project_id=097213bd8ea048a6b079a72065d5f1f4)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:26.171580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570440ed-f973-4c51-969f-3b0cd5cc5a33 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:26.173319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 559/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:26 2022] GET /identity/v3/projects/097213bd8ea048a6b079a72065d5f1f4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:26.179890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:26.201814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:26.280676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820913d-e10e-4cec-b74c-bc456f1c3246 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:26.314356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dd68d-6d2e-4f45-96ef-2602a8fb584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:26.315000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd68d-6d2e-4f45-96ef-2602a8fb584c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:26.315281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd68d-6d2e-4f45-96ef-2602a8fb584c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:26.315558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd68d-6d2e-4f45-96ef-2602a8fb584c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:26.316372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd68d-6d2e-4f45-96ef-2602a8fb584c None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=097213bd8ea048a6b079a72065d5f1f4, user.id=17265851b6ba4f088414a36bec764346) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:26.316711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd68d-6d2e-4f45-96ef-2602a8fb584c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:26.318128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd68d-6d2e-4f45-96ef-2602a8fb584c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:26.323410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 587/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:26 2022] GET /identity/v3/role_assignments?user.id=17265851b6ba4f088414a36bec764346&scope.project.id=097213bd8ea048a6b079a72065d5f1f4&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:28.346102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549eeaa-3f6d-4742-8d4c-1e15604d0b69 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:28.346623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549eeaa-3f6d-4742-8d4c-1e15604d0b69 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:28.346623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549eeaa-3f6d-4742-8d4c-1e15604d0b69 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:28.347932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 560/1147] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:28.353258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4f5e-db21-4e43-b793-0f91eb9a449c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:28.353682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4f5e-db21-4e43-b793-0f91eb9a449c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:28.353813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da4f5e-db21-4e43-b793-0f91eb9a449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:28.371132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d2da4f5e-db21-4e43-b793-0f91eb9a449c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:28.372305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d2da4f5e-db21-4e43-b793-0f91eb9a449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:28.439929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 588/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:28.463364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549eeaa-3f6d-4742-8d4c-1e15604d0b69 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:28.567473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549eeaa-3f6d-4742-8d4c-1e15604d0b69 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:28.577353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:28.602322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:28.603038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:28.603322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:28.603598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:28.613454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:28.621625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:28.622731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:28.623611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 561/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:28.629073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da4f5e-db21-4e43-b793-0f91eb9a449c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:28.643028 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:28.726265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da4f5e-db21-4e43-b793-0f91eb9a449c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:28.759535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:28.760197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:28.760476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:28.760777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:28.761516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:28.761822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:28.763180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:28.771383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 589/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:28.777221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:28.876819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb273b-aa6d-4b9c-a8af-3a1b0fbbb61f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:28.909537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:28.910403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:28.910686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:28.910960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] PATH_INFO: `/v3/users/17265851b6ba4f088414a36bec764346` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:28.912142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] RBAC: Authorizing `identity:get_user(user_id=17265851b6ba4f088414a36bec764346)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:28.914152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:28.915771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 562/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:28 2022] GET /identity/v3/users/17265851b6ba4f088414a36bec764346 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:28.921526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:28.994408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:29.018955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcc895-61f2-4cc1-bd6b-d2a189ee7e64 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:29.053449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654389f3-42d2-4f35-ae65-e95373cdb862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:29.054321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654389f3-42d2-4f35-ae65-e95373cdb862 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:29.054672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654389f3-42d2-4f35-ae65-e95373cdb862 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:29.054994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654389f3-42d2-4f35-ae65-e95373cdb862 None admin] PATH_INFO: `/v3/projects/097213bd8ea048a6b079a72065d5f1f4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:29.056266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654389f3-42d2-4f35-ae65-e95373cdb862 None admin] RBAC: Authorizing `identity:get_project(project_id=097213bd8ea048a6b079a72065d5f1f4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:29.057750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654389f3-42d2-4f35-ae65-e95373cdb862 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:29.059727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 590/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:28 2022] GET /identity/v3/projects/097213bd8ea048a6b079a72065d5f1f4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:29.065783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:29.100610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:29.164972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e2f0c-e851-4ed2-9db1-8af962dddfb2 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:29.197703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fbc6c-8755-4a13-bde7-b5690b583080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:29.198617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fbc6c-8755-4a13-bde7-b5690b583080 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:29.198900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fbc6c-8755-4a13-bde7-b5690b583080 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:29.199178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fbc6c-8755-4a13-bde7-b5690b583080 None admin] PATH_INFO: `/v3/projects/097213bd8ea048a6b079a72065d5f1f4/users/17265851b6ba4f088414a36bec764346/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:29.201139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fbc6c-8755-4a13-bde7-b5690b583080 None admin] RBAC: Authorizing `identity:create_grant(project_id=097213bd8ea048a6b079a72065d5f1f4, user_id=17265851b6ba4f088414a36bec764346, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:29.202567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fbc6c-8755-4a13-bde7-b5690b583080 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:29.215791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 563/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:07:29 2022] PUT /identity/v3/projects/097213bd8ea048a6b079a72065d5f1f4/users/17265851b6ba4f088414a36bec764346/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:07:31.251838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a637-b6d9-4856-af13-9e3240e35a24 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:31.252362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a637-b6d9-4856-af13-9e3240e35a24 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:31.252744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a637-b6d9-4856-af13-9e3240e35a24 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:31.254831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 591/1154] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:31.260660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba2ce-998f-4796-823e-74497a667e4d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:31.261086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba2ce-998f-4796-823e-74497a667e4d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:31.261211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba2ce-998f-4796-823e-74497a667e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:31.277759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1a8ba2ce-998f-4796-823e-74497a667e4d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:31.278921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1a8ba2ce-998f-4796-823e-74497a667e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:31.346022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 564/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:31.370183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651a637-b6d9-4856-af13-9e3240e35a24 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:31.489031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651a637-b6d9-4856-af13-9e3240e35a24 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:31.526727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:31.527517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:31.527804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:31.528082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:31.539630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:31.547525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:31.548708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:31.549917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 592/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:31.555959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba2ce-998f-4796-823e-74497a667e4d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:31.588585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:31.662184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba2ce-998f-4796-823e-74497a667e4d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:31.701826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:31.702484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:31.702764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:31.703037 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:31.703772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:31.704076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:31.705455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:31.713297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 565/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:31.719145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:31.722022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:31.818054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b406db4-a20c-427d-b316-dc9f2b99284d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:31.851876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:31.852774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:31.853063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:31.853343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] PATH_INFO: `/v3/users/17265851b6ba4f088414a36bec764346` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:31.854546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] RBAC: Authorizing `identity:get_user(user_id=17265851b6ba4f088414a36bec764346)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:31.855984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:31.857662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 593/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:31 2022] GET /identity/v3/users/17265851b6ba4f088414a36bec764346 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:31.863392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:31.948731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:31.961091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6fc4a-9972-4fd0-823d-744319be3e6b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:31.993566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:31.994210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:31.994493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:31.994771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:32.060131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:32.061599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:32.144528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:32.158054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 566/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:32.164519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:32.231796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:32.261800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876e8ca-9115-440c-96b9-996d6f5cb0cf None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:32.295131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2edfee-1271-41ff-92a5-664c0bf2d85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:32.295981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2edfee-1271-41ff-92a5-664c0bf2d85c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:32.296267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2edfee-1271-41ff-92a5-664c0bf2d85c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:32.296546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2edfee-1271-41ff-92a5-664c0bf2d85c None admin] PATH_INFO: `/v3/projects/097213bd8ea048a6b079a72065d5f1f4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:32.297755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2edfee-1271-41ff-92a5-664c0bf2d85c None admin] RBAC: Authorizing `identity:get_project(project_id=097213bd8ea048a6b079a72065d5f1f4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:32.299182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2edfee-1271-41ff-92a5-664c0bf2d85c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:32.300937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 594/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:32 2022] GET /identity/v3/projects/097213bd8ea048a6b079a72065d5f1f4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:32.307044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:32.403886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1fd82-9065-45f3-943f-d87637aab239 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:32.437494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b9b10-9dbd-45f4-90d4-a884d46d0a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:32.438144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9b10-9dbd-45f4-90d4-a884d46d0a3e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:32.438429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9b10-9dbd-45f4-90d4-a884d46d0a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:32.438706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9b10-9dbd-45f4-90d4-a884d46d0a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:32.439528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b9b10-9dbd-45f4-90d4-a884d46d0a3e None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=097213bd8ea048a6b079a72065d5f1f4, user.id=17265851b6ba4f088414a36bec764346) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:32.439840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b9b10-9dbd-45f4-90d4-a884d46d0a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:32.441285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b9b10-9dbd-45f4-90d4-a884d46d0a3e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:32.447059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 567/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:32 2022] GET /identity/v3/role_assignments?user.id=17265851b6ba4f088414a36bec764346&scope.project.id=097213bd8ea048a6b079a72065d5f1f4&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:34.474405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b25e7-8c14-4b88-9b72-edc3277aa925 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:34.474926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b25e7-8c14-4b88-9b72-edc3277aa925 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:34.474926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b25e7-8c14-4b88-9b72-edc3277aa925 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:34.476803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 595/1162] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:34.481967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a9c4-b7d3-4203-938e-230d17af0c1a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:34.482347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a9c4-b7d3-4203-938e-230d17af0c1a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:34.482528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a9c4-b7d3-4203-938e-230d17af0c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:34.499429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5e40a9c4-b7d3-4203-938e-230d17af0c1a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:34.500636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5e40a9c4-b7d3-4203-938e-230d17af0c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:34.549529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:34.569691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 568/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:34.593184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b25e7-8c14-4b88-9b72-edc3277aa925 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:34.693597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b25e7-8c14-4b88-9b72-edc3277aa925 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:34.728380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:34.729101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:34.729384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:34.729662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:34.737427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:34.795661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:34.797114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:34.893136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 596/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:34.899160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40a9c4-b7d3-4203-938e-230d17af0c1a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:34.993083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40a9c4-b7d3-4203-938e-230d17af0c1a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:35.025542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fddbae-e2a6-4af2-9c2c-e74a4de8c191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:35.026326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fddbae-e2a6-4af2-9c2c-e74a4de8c191 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:35.026604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fddbae-e2a6-4af2-9c2c-e74a4de8c191 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:35.026874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fddbae-e2a6-4af2-9c2c-e74a4de8c191 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:35.033776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fddbae-e2a6-4af2-9c2c-e74a4de8c191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:35.035545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fddbae-e2a6-4af2-9c2c-e74a4de8c191 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-81fddbae-e2a6-4af2-9c2c-e74a4de8c191 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:35.049433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:35.050660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:35.051720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 569/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:07:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:35.056994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:35.095381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:35.153432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f77c0f-98b1-4e56-9fb2-a81717521781 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:35.186762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0d004-53c2-4983-9555-778f49d73ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:35.187416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d004-53c2-4983-9555-778f49d73ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:35.187701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d004-53c2-4983-9555-778f49d73ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:35.187976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d004-53c2-4983-9555-778f49d73ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:35.188716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0d004-53c2-4983-9555-778f49d73ec0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:35.189021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0d004-53c2-4983-9555-778f49d73ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:35.190370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0d004-53c2-4983-9555-778f49d73ec0 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:35.198695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 597/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Jun 24 09:07:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:37.219349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2092-01d1-4dcc-8fe6-ff9c1fbead2b None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:37.219936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2092-01d1-4dcc-8fe6-ff9c1fbead2b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:37.219936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2092-01d1-4dcc-8fe6-ff9c1fbead2b None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:37.221302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 570/1167] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:37.226619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34e79d-2e02-4b0a-9cdf-aaab6c7aa1d1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:37.227050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34e79d-2e02-4b0a-9cdf-aaab6c7aa1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:37.227175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34e79d-2e02-4b0a-9cdf-aaab6c7aa1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:37.247360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0a34e79d-2e02-4b0a-9cdf-aaab6c7aa1d1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:37.248553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0a34e79d-2e02-4b0a-9cdf-aaab6c7aa1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:37.314308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 598/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:37.338279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f2092-01d1-4dcc-8fe6-ff9c1fbead2b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:37.382798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:37.451633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f2092-01d1-4dcc-8fe6-ff9c1fbead2b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:37.486911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ca5ae-5044-4586-a3db-308873a57eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:37.487573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca5ae-5044-4586-a3db-308873a57eac None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:37.487856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca5ae-5044-4586-a3db-308873a57eac None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:37.488134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca5ae-5044-4586-a3db-308873a57eac None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:37.488906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ca5ae-5044-4586-a3db-308873a57eac None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:37.490280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ca5ae-5044-4586-a3db-308873a57eac None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:37.506395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 571/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Fri Jun 24 09:07:37 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:07:39.584095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7763f-ce79-4529-8a49-f731ee9cec2d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:39.584712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7763f-ce79-4529-8a49-f731ee9cec2d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:39.584712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7763f-ce79-4529-8a49-f731ee9cec2d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:39.586584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 599/1170] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:39.591810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421fb0a-9d8a-4a08-aa7b-60b370d10f10 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:39.592268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421fb0a-9d8a-4a08-aa7b-60b370d10f10 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:39.592360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421fb0a-9d8a-4a08-aa7b-60b370d10f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:39.610381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c421fb0a-9d8a-4a08-aa7b-60b370d10f10 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:39.611852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c421fb0a-9d8a-4a08-aa7b-60b370d10f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:39.679461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 572/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:39.703427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7763f-ce79-4529-8a49-f731ee9cec2d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:39.737128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:39.810878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7763f-ce79-4529-8a49-f731ee9cec2d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:39.845691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608f07c-aa53-450c-bca7-2d9a39371e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:39.846479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608f07c-aa53-450c-bca7-2d9a39371e50 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:39.846760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608f07c-aa53-450c-bca7-2d9a39371e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:39.847032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608f07c-aa53-450c-bca7-2d9a39371e50 None admin] PATH_INFO: `/v3/domains/47a32fba0cc4409189d668490d036d14` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:39.859690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608f07c-aa53-450c-bca7-2d9a39371e50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=47a32fba0cc4409189d668490d036d14)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:39.862388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608f07c-aa53-450c-bca7-2d9a39371e50 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:39.865470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 600/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Fri Jun 24 09:07:39 2022] GET /identity/v3/domains/47a32fba0cc4409189d668490d036d14 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:39.871441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421fb0a-9d8a-4a08-aa7b-60b370d10f10 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:39.906355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:39.977662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421fb0a-9d8a-4a08-aa7b-60b370d10f10 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:40.011381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77345f-00f2-4442-a349-1615e4fcc62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:40.012053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77345f-00f2-4442-a349-1615e4fcc62a None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:40.012335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77345f-00f2-4442-a349-1615e4fcc62a None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:40.012634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77345f-00f2-4442-a349-1615e4fcc62a None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:40.013492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77345f-00f2-4442-a349-1615e4fcc62a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:40.014938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77345f-00f2-4442-a349-1615e4fcc62a None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:40.044682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 573/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:07:39 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:07:42.135517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4878e34-487f-4e86-b78e-134227a478b6 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:42.136068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4878e34-487f-4e86-b78e-134227a478b6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:42.136068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4878e34-487f-4e86-b78e-134227a478b6 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:42.137466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 601/1174] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:42.142727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b484a0-c98f-44b0-af50-555c1a036dc4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:42.143125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b484a0-c98f-44b0-af50-555c1a036dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:42.143287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b484a0-c98f-44b0-af50-555c1a036dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:42.160437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-25b484a0-c98f-44b0-af50-555c1a036dc4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:42.161585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-25b484a0-c98f-44b0-af50-555c1a036dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:42.231800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 574/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:42.256303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4878e34-487f-4e86-b78e-134227a478b6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:42.361222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:42.371648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4878e34-487f-4e86-b78e-134227a478b6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:42.405540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb1b13-e6f6-439c-addc-4aa62038d8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:42.406400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb1b13-e6f6-439c-addc-4aa62038d8ac None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:42.406683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb1b13-e6f6-439c-addc-4aa62038d8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:42.406960 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb1b13-e6f6-439c-addc-4aa62038d8ac None admin] PATH_INFO: `/v3/domains/47a32fba0cc4409189d668490d036d14` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:42.408150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb1b13-e6f6-439c-addc-4aa62038d8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=47a32fba0cc4409189d668490d036d14)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:42.409616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb1b13-e6f6-439c-addc-4aa62038d8ac None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:42.411243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 602/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Fri Jun 24 09:07:42 2022] GET /identity/v3/domains/47a32fba0cc4409189d668490d036d14 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:42.418424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b484a0-c98f-44b0-af50-555c1a036dc4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:42.441756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:42.527305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b484a0-c98f-44b0-af50-555c1a036dc4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:42.561690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8e2d0-d87c-4ae5-8f99-ef80f20dd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:42.562359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8e2d0-d87c-4ae5-8f99-ef80f20dd226 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:42.562727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8e2d0-d87c-4ae5-8f99-ef80f20dd226 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:42.562899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8e2d0-d87c-4ae5-8f99-ef80f20dd226 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:42.563607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8e2d0-d87c-4ae5-8f99-ef80f20dd226 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:42.565083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8e2d0-d87c-4ae5-8f99-ef80f20dd226 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:42.585463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 575/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:07:42 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:07:44.680894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848786-25ea-42af-ba5e-a92443bb2251 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:44.681446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848786-25ea-42af-ba5e-a92443bb2251 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:44.681446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848786-25ea-42af-ba5e-a92443bb2251 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:44.682806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 603/1178] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:44.688300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99d9f-e68c-482b-8864-418aa81af0fe None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:44.688755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99d9f-e68c-482b-8864-418aa81af0fe None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:44.688881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99d9f-e68c-482b-8864-418aa81af0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:44.706310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9df99d9f-e68c-482b-8864-418aa81af0fe None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:44.707472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9df99d9f-e68c-482b-8864-418aa81af0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:44.767211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:44.772439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 576/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:44.796427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848786-25ea-42af-ba5e-a92443bb2251 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:44.894176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848786-25ea-42af-ba5e-a92443bb2251 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:44.927328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:44.928107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:44.928390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:44.928707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:44.939206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:44.947154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:44.948312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:44.949441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 604/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:44.954279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df99d9f-e68c-482b-8864-418aa81af0fe None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:45.049191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df99d9f-e68c-482b-8864-418aa81af0fe None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:45.082140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:45.082798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:45.083080 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:45.083353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:45.084091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:45.084396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:45.086400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:45.094247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 577/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:45.102074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:45.124511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:45.201242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb491c-6bca-499b-b50e-38cb97842e8e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:45.234016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:45.234874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:45.235151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:45.235418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] PATH_INFO: `/v3/users/5b4a032c9a4c4d98a1d9b213ab2500c5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:45.248035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4a032c9a4c4d98a1d9b213ab2500c5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:45.249481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:45.251089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 605/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:45 2022] GET /identity/v3/users/5b4a032c9a4c4d98a1d9b213ab2500c5 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:45.256853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:45.318020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:45.357285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12897afb-8481-40bd-baa5-e3d57cec7702 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:45.390126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:45.390771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:45.391060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:45.391340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:45.456699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:45.458074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:45.518981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:45.557761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 578/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:07:45.563997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:45.639644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:45.660809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e091b0-045a-472e-8bd2-0ae71af42447 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:45.697130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dd683-3f3f-4870-930c-874382790c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:45.698114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dd683-3f3f-4870-930c-874382790c22 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:45.698400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dd683-3f3f-4870-930c-874382790c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:45.698784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dd683-3f3f-4870-930c-874382790c22 None admin] PATH_INFO: `/v3/projects/6c608a0ad7b3422897529b6284ae722d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:45.700247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dd683-3f3f-4870-930c-874382790c22 None admin] RBAC: Authorizing `identity:get_project(project_id=6c608a0ad7b3422897529b6284ae722d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:45.701700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dd683-3f3f-4870-930c-874382790c22 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:45.703404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 606/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:45 2022] GET /identity/v3/projects/6c608a0ad7b3422897529b6284ae722d => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:45.709966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:45.814471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b3476-d8a4-49a4-afe1-e6e0e2766975 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:45.848462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c20d08-61f9-4b6b-8985-3137858658e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:45.849147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c20d08-61f9-4b6b-8985-3137858658e1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:45.849434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c20d08-61f9-4b6b-8985-3137858658e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:45.849716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c20d08-61f9-4b6b-8985-3137858658e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:45.850545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c20d08-61f9-4b6b-8985-3137858658e1 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=6c608a0ad7b3422897529b6284ae722d, user.id=5b4a032c9a4c4d98a1d9b213ab2500c5) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:45.850858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c20d08-61f9-4b6b-8985-3137858658e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:45.852861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c20d08-61f9-4b6b-8985-3137858658e1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:45.858585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 579/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:45 2022] GET /identity/v3/role_assignments?user.id=5b4a032c9a4c4d98a1d9b213ab2500c5&scope.project.id=6c608a0ad7b3422897529b6284ae722d&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:07:47.873951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05181256-d58d-433f-9d15-7fa8f8106703 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:47.874495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05181256-d58d-433f-9d15-7fa8f8106703 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:47.874495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05181256-d58d-433f-9d15-7fa8f8106703 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:47.875853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 607/1186] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:47.881150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec40f-55db-4428-88c4-d2394f0e6fc7 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:47.881556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec40f-55db-4428-88c4-d2394f0e6fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:47.881705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec40f-55db-4428-88c4-d2394f0e6fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:47.899051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c6aec40f-55db-4428-88c4-d2394f0e6fc7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:47.900250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c6aec40f-55db-4428-88c4-d2394f0e6fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:47.921624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:47.970352 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 580/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:47.992892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05181256-d58d-433f-9d15-7fa8f8106703 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:48.105194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05181256-d58d-433f-9d15-7fa8f8106703 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:48.139167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:48.139878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:48.140165 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:48.140441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:48.151101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:48.158718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:48.159885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:07:48.163118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 608/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:07:48.166389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec40f-55db-4428-88c4-d2394f0e6fc7 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:48.260678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec40f-55db-4428-88c4-d2394f0e6fc7 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:48.292941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:48.293594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:48.293875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:48.294149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:48.294880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:48.295179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:48.296527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:48.304333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 581/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:48.310420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:48.319597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:48.409974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d4ffc-c7c0-42fd-911d-b743bd29a40f None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:48.442565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:48.443427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:48.443711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:48.443988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] PATH_INFO: `/v3/users/5b4a032c9a4c4d98a1d9b213ab2500c5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:48.445796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4a032c9a4c4d98a1d9b213ab2500c5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:48.447217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:48.448858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 609/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:48 2022] GET /identity/v3/users/5b4a032c9a4c4d98a1d9b213ab2500c5 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:48.454625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:48.473461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:48.550631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7db39-ffd3-4961-8745-0847db2443a1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:48.583355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c98bb9-2974-44b9-b02f-b67c1edead30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:48.584195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c98bb9-2974-44b9-b02f-b67c1edead30 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:48.584471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c98bb9-2974-44b9-b02f-b67c1edead30 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:48.584769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c98bb9-2974-44b9-b02f-b67c1edead30 None admin] PATH_INFO: `/v3/projects/6c608a0ad7b3422897529b6284ae722d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:48.585918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c98bb9-2974-44b9-b02f-b67c1edead30 None admin] RBAC: Authorizing `identity:get_project(project_id=6c608a0ad7b3422897529b6284ae722d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:48.587329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c98bb9-2974-44b9-b02f-b67c1edead30 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:48.589026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 582/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:48 2022] GET /identity/v3/projects/6c608a0ad7b3422897529b6284ae722d => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:48.594894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:48.667605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:48.692596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae488af-3b5a-4aad-ac1e-a9be89d4e556 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:48.724805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d2c4-83e4-403f-bc56-bec57abd33f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:48.725683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d2c4-83e4-403f-bc56-bec57abd33f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:48.726486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d2c4-83e4-403f-bc56-bec57abd33f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:48.726761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d2c4-83e4-403f-bc56-bec57abd33f5 None admin] PATH_INFO: `/v3/projects/6c608a0ad7b3422897529b6284ae722d/users/5b4a032c9a4c4d98a1d9b213ab2500c5/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:48.728713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6d2c4-83e4-403f-bc56-bec57abd33f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c608a0ad7b3422897529b6284ae722d, user_id=5b4a032c9a4c4d98a1d9b213ab2500c5, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:48.730167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6d2c4-83e4-403f-bc56-bec57abd33f5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:48.742930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 610/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:07:48 2022] PUT /identity/v3/projects/6c608a0ad7b3422897529b6284ae722d/users/5b4a032c9a4c4d98a1d9b213ab2500c5/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:07:50.750357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939968c-a5d9-470a-bba0-37704ac94183 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:50.750895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939968c-a5d9-470a-bba0-37704ac94183 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:50.750895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939968c-a5d9-470a-bba0-37704ac94183 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:50.752389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 583/1193] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:07:50.757860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65767fc5-946f-4951-924f-0b8dc22d6b14 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:50.758296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65767fc5-946f-4951-924f-0b8dc22d6b14 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:50.758405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65767fc5-946f-4951-924f-0b8dc22d6b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:50.775417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-65767fc5-946f-4951-924f-0b8dc22d6b14 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:07:50.776627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-65767fc5-946f-4951-924f-0b8dc22d6b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:07:50.847806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 611/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:07:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:07:50.872426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939968c-a5d9-470a-bba0-37704ac94183 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:50.960059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:50.977952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939968c-a5d9-470a-bba0-37704ac94183 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:51.012370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:51.013115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:51.013393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:51.013670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:51.024860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:07:51.032441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:07:51.033400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:07:51.034368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 584/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:07:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:07:51.041540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65767fc5-946f-4951-924f-0b8dc22d6b14 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:51.104744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:51.153226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65767fc5-946f-4951-924f-0b8dc22d6b14 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:51.188293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:51.188991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:51.189273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:51.189546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:51.190290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:51.190592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:51.191948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:51.201008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 612/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:07:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:51.206812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:51.307038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a5086-e370-4aed-b3c1-df776768dfa8 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:51.339760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:51.340643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:51.340922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:51.341193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] PATH_INFO: `/v3/users/5b4a032c9a4c4d98a1d9b213ab2500c5` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:51.342344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] RBAC: Authorizing `identity:get_user(user_id=5b4a032c9a4c4d98a1d9b213ab2500c5)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:51.343763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:51.345383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 585/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Jun 24 09:07:51 2022] GET /identity/v3/users/5b4a032c9a4c4d98a1d9b213ab2500c5 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:51.351109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:51.449699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cd10f-68b4-477d-870b-f4216cb08e38 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:51.473588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:51.482295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:51.482928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:51.483208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:51.483479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:51.548144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:51.549606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:51.648339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 613/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:07:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:07:51.654097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:51.669978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:51.754122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acb253-0741-4e0a-8984-13dd7b989bbe None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:51.787866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197ed24-48a1-4548-a058-ceec0f4fb062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:51.788907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ed24-48a1-4548-a058-ceec0f4fb062 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:51.789194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ed24-48a1-4548-a058-ceec0f4fb062 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:51.789475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ed24-48a1-4548-a058-ceec0f4fb062 None admin] PATH_INFO: `/v3/projects/6c608a0ad7b3422897529b6284ae722d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:51.790643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197ed24-48a1-4548-a058-ceec0f4fb062 None admin] RBAC: Authorizing `identity:get_project(project_id=6c608a0ad7b3422897529b6284ae722d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:51.792099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197ed24-48a1-4548-a058-ceec0f4fb062 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:51.793824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 586/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Jun 24 09:07:51 2022] GET /identity/v3/projects/6c608a0ad7b3422897529b6284ae722d => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:07:51.799905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:07:51.826625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:07:51.899953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c184021-2641-4e9b-b0d7-b7172310ebdd None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:07:51.933941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa6f19-8707-4a0f-b1df-4c74aff67de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:07:51.934705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa6f19-8707-4a0f-b1df-4c74aff67de5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:07:51.934987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa6f19-8707-4a0f-b1df-4c74aff67de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:07:51.935262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa6f19-8707-4a0f-b1df-4c74aff67de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:07:51.936073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa6f19-8707-4a0f-b1df-4c74aff67de5 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=6c608a0ad7b3422897529b6284ae722d, user.id=5b4a032c9a4c4d98a1d9b213ab2500c5) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:07:51.936377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa6f19-8707-4a0f-b1df-4c74aff67de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:07:51.937815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa6f19-8707-4a0f-b1df-4c74aff67de5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:07:51.943438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 614/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:07:51 2022] GET /identity/v3/role_assignments?user.id=5b4a032c9a4c4d98a1d9b213ab2500c5&scope.project.id=6c608a0ad7b3422897529b6284ae722d&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:01.783835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=59 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:01.940903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=55 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:47.729680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358bb47-32ca-4014-abb4-70d44217d5eb None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:47.730320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358bb47-32ca-4014-abb4-70d44217d5eb None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:47.730320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358bb47-32ca-4014-abb4-70d44217d5eb None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:47.731883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 587/1201] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:08:47.739837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e45c03-55f3-4bd6-8eb7-677f80a85653 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:47.739837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e45c03-55f3-4bd6-8eb7-677f80a85653 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:47.739837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e45c03-55f3-4bd6-8eb7-677f80a85653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:47.758444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-56e45c03-55f3-4bd6-8eb7-677f80a85653 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:08:47.760386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-56e45c03-55f3-4bd6-8eb7-677f80a85653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:08:47.872803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 615/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:08:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:08:47.893306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358bb47-32ca-4014-abb4-70d44217d5eb None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:48.045761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358bb47-32ca-4014-abb4-70d44217d5eb None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:48.091930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:48.092999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:48.093317 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:48.093638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:48.109341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:48.110927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:08:48.134478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:48.135954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:48.137054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:08:48.137054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:08:48.137054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:48.137054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:08:48.141635 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 588/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:48.143423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e45c03-55f3-4bd6-8eb7-677f80a85653 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:48.284603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e45c03-55f3-4bd6-8eb7-677f80a85653 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:48.330282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:48.339143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11889-0b38-4d5e-9d2e-bf31fcddbea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:48.340052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11889-0b38-4d5e-9d2e-bf31fcddbea2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:48.340327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11889-0b38-4d5e-9d2e-bf31fcddbea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:48.340605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11889-0b38-4d5e-9d2e-bf31fcddbea2 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:48.341384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11889-0b38-4d5e-9d2e-bf31fcddbea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:48.341832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11889-0b38-4d5e-9d2e-bf31fcddbea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:48.343614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11889-0b38-4d5e-9d2e-bf31fcddbea2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:48.360537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 616/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:48.366498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:48.445524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:48.516582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ddea-7287-40f6-b26e-8edd56a9a9b4 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:48.572908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8653ac-f4ca-4ad4-bc4a-8cd36e341593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:48.573739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8653ac-f4ca-4ad4-bc4a-8cd36e341593 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:48.574069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8653ac-f4ca-4ad4-bc4a-8cd36e341593 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:48.574398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8653ac-f4ca-4ad4-bc4a-8cd36e341593 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:48.575500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8653ac-f4ca-4ad4-bc4a-8cd36e341593 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:48.577328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8653ac-f4ca-4ad4-bc4a-8cd36e341593 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:48.608277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 589/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Jun 24 09:08:48 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:08:50.975757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3f9f-f8a7-404d-b99e-6c30adf7d2f4 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:50.976336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3f9f-f8a7-404d-b99e-6c30adf7d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:50.976336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3f9f-f8a7-404d-b99e-6c30adf7d2f4 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:50.977723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 617/1206] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:08:50.987995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc69d-6e1f-442d-ad41-2646e976bff3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:50.988662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc69d-6e1f-442d-ad41-2646e976bff3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:50.988891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc69d-6e1f-442d-ad41-2646e976bff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:51.007489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3b7dc69d-6e1f-442d-ad41-2646e976bff3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:08:51.008742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3b7dc69d-6e1f-442d-ad41-2646e976bff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:08:51.104596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 590/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:08:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:08:51.127356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed3f9f-f8a7-404d-b99e-6c30adf7d2f4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:51.254058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed3f9f-f8a7-404d-b99e-6c30adf7d2f4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:51.294244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:51.295040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:51.295320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:51.295632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:51.308434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:08:51.318083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:51.319121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:51.319983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 618/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:08:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:51.326686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dc69d-6e1f-442d-ad41-2646e976bff3 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:51.374857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:51.449329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dc69d-6e1f-442d-ad41-2646e976bff3 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:51.494079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:51.494759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:51.495054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:51.495322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:51.496166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:51.496480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:51.497979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:51.506718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 591/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:08:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:51.513216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:51.573058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:51.630407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82edb56-6ff4-47fc-9500-32f1e1420acc None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:51.670614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:51.674806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:51.674806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:51.674806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:51.685626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:51.687122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:08:51.706767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:51.707866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:51.709245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:08:51.709245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:08:51.709245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:51.709245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:51.709245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 619/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:51.714786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:51.837113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:51.840516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd4dbb-4bde-42c6-8d71-e7ab4f1d8c50 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:51.878236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:51.879076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:51.879473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:51.879839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:51.880679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:51.881055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:51.882482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:51.897629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 592/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:51.903723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:52.050113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e1fb-0c38-4d29-9495-bfc2792b45e7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:52.052623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:52.095140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:52.096000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:52.096337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:52.096641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:52.105268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:52.107013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:08:52.122843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:52.123941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:52.129686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 620/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:08:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:52.131501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:52.256389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566c3da-d735-4696-a7ee-ab7282745e95 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:52.294807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:52.295647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:52.296030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:52.296425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:52.297393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:52.297804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:52.299331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:52.318677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 593/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Jun 24 09:08:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:52.324750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:52.454098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fa21a-3bdc-4f88-b2da-e26ee6f2b3aa None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:52.495410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:52.496243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:52.496532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:52.496833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:52.537840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:52.585994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:52.587718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:52.714349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 621/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:08:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:08:52.720489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:52.747150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:52.840088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b487f79-c4a9-43bc-adad-df79e82664ba None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:52.883046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:52.883911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:52.884220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:52.884618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:52.898444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:52.900203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:08:52.919545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:52.920746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:52.921889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:08:52.921889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:08:52.921889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:52.921889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:08:52.926683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 594/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:08:52.927726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:52.998998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:53.038827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ccf6d-e116-44d3-aad2-d80b11bfd452 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:53.075553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:53.076206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:53.076482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:53.076790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:53.077515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:53.077818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:53.079220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:53.095721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 622/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:08:53.097424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:53.187730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:53.226801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a4a1e-3d23-4fe3-9d39-d2c07c18eaf5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:53.267366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:53.268170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:53.268529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:53.268771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:53.279921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:53.281558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:08:53.297750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:53.298947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:08:53.299890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 595/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:08:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:08:53.305350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:53.431420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29beade0-73a2-4d77-8caa-3a1cbe4fed3c None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:53.456868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:53.473748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019c2c2-fcc6-49b0-936a-a9e2c31f03b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:53.474745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019c2c2-fcc6-49b0-936a-a9e2c31f03b5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:53.475096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019c2c2-fcc6-49b0-936a-a9e2c31f03b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:53.475657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019c2c2-fcc6-49b0-936a-a9e2c31f03b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:53.476959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019c2c2-fcc6-49b0-936a-a9e2c31f03b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:53.477345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019c2c2-fcc6-49b0-936a-a9e2c31f03b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:53.479502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019c2c2-fcc6-49b0-936a-a9e2c31f03b5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:53.503960 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 623/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:53.505846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:53.626548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109bc4b-826d-4667-a8a2-1d71128e685b None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:53.659613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:53.667379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133809a-6a71-49e9-a38f-ef65790660f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:53.668048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133809a-6a71-49e9-a38f-ef65790660f3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:53.668331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133809a-6a71-49e9-a38f-ef65790660f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:53.668630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133809a-6a71-49e9-a38f-ef65790660f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:53.669461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133809a-6a71-49e9-a38f-ef65790660f3 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=505dda787dac43bca3595fa1c406a457) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:53.669783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133809a-6a71-49e9-a38f-ef65790660f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:53.671246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133809a-6a71-49e9-a38f-ef65790660f3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:53.684617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 596/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:08:53 2022] GET /identity/v3/role_assignments?user.id=505dda787dac43bca3595fa1c406a457&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:08:55.844080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dcfcf-9cf6-4000-953a-97a296187ecf None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:55.844660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dcfcf-9cf6-4000-953a-97a296187ecf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:55.844660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dcfcf-9cf6-4000-953a-97a296187ecf None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:55.846398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 624/1220] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:08:55.852034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcb1d-1583-4c49-a477-01a83ec02f88 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:55.852488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcb1d-1583-4c49-a477-01a83ec02f88 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:55.852663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcb1d-1583-4c49-a477-01a83ec02f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:55.878095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-16dfcb1d-1583-4c49-a477-01a83ec02f88 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:08:55.879292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-16dfcb1d-1583-4c49-a477-01a83ec02f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:08:55.965174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 597/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:08:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:08:55.988398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dcfcf-9cf6-4000-953a-97a296187ecf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:56.105913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dcfcf-9cf6-4000-953a-97a296187ecf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:56.145690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:56.146629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:56.146952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:56.147305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:56.161244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:08:56.169385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:56.170546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:56.176778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 625/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:08:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:56.179016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfcb1d-1583-4c49-a477-01a83ec02f88 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:56.286133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfcb1d-1583-4c49-a477-01a83ec02f88 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:56.312736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:56.326092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:56.326766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:56.327047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:56.327323 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:56.328069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:56.328380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:56.329776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:56.338445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 598/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:08:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:56.344272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:56.390701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:56.460186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d8e84-72c1-4de7-bbaa-e1082ca7a569 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:56.498128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:56.498898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:56.499176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:56.499455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:56.511279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:56.512871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:08:56.531075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:56.532217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:56.533254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:08:56.533254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:08:56.533254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:56.533254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:56.533254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 626/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:56.538108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:56.645908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b5047-2c2b-4e4f-b5fb-e7ed72e3b6b5 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:56.683823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:56.684786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:56.685070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:56.685346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:56.686062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:56.686370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:56.687729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:56.701719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 599/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:56.707429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:56.805743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:56.815292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18192ab-eedf-46ba-bfd3-48475feef3ba None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:56.856978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:56.857997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:56.858308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:56.858623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:56.866942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:56.868371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:08:56.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:56.885122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:56.891037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 627/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:08:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:56.891889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:56.944648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:57.006444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73a248-25fa-450c-a897-cf55f41ab4ae None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:57.042800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:57.043479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:57.043765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:57.044031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:57.044907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:57.045208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:57.046636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:57.073615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 600/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Jun 24 09:08:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:57.074927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:57.192198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203cdab4-f36d-4c70-be57-e374e8b6c674 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:57.233569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:57.234294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:57.234576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:57.234848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:57.247202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:57.248983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:08:57.270146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:57.271279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:57.272595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:08:57.272595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:08:57.272595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:08:57.272595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:57.272595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 628/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:08:57.277329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:57.388914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e89be-0deb-4320-a924-16073b47eb29 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:57.391261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:57.431249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:57.431916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:57.432195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:57.432469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:57.433230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:57.433534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:57.435755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:57.451885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 601/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:57.458467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:57.470066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:57.585815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c75b32-c4bf-492c-8cc1-385d32b88b27 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:57.624372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:57.625349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:57.625736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:57.625972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:57.633943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:57.636318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:08:57.653453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:08:57.654649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:08:57.655576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 629/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:08:57.660909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:57.771066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf3b24-866f-46c5-b68a-a15fc9e61148 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:57.807088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330221a9-1d91-4b93-8364-b9fa98f60c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:57.807786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330221a9-1d91-4b93-8364-b9fa98f60c74 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:57.808083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330221a9-1d91-4b93-8364-b9fa98f60c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:57.808381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330221a9-1d91-4b93-8364-b9fa98f60c74 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:57.809557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330221a9-1d91-4b93-8364-b9fa98f60c74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:08:57.809865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330221a9-1d91-4b93-8364-b9fa98f60c74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:57.811384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330221a9-1d91-4b93-8364-b9fa98f60c74 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:57.825537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 602/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:08:57.835739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:08:57.921989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:08:57.965654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1f57-8f5d-4fb0-b47f-96b3bb46de81 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:08:58.008017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c2d85-d00e-48dc-9973-d7407e965f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:08:58.008948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2d85-d00e-48dc-9973-d7407e965f36 None admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:08:58.009779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2d85-d00e-48dc-9973-d7407e965f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:08:58.010036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2d85-d00e-48dc-9973-d7407e965f36 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/505dda787dac43bca3595fa1c406a457/roles/9654f3986243401db4f1aad536273ede` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:08:58.029910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c2d85-d00e-48dc-9973-d7407e965f36 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=505dda787dac43bca3595fa1c406a457, role_id=9654f3986243401db4f1aad536273ede)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:08:58.031996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c2d85-d00e-48dc-9973-d7407e965f36 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:08:58.050157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 630/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:08:57 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/505dda787dac43bca3595fa1c406a457/roles/9654f3986243401db4f1aad536273ede => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:09:00.284247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fcc2a-7dfc-4580-b164-c166183be84b None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:00.284980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fcc2a-7dfc-4580-b164-c166183be84b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:00.285506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fcc2a-7dfc-4580-b164-c166183be84b None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:00.288062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 603/1233] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:09:00.294023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54df4-6456-4cc8-bf9a-5ef7fa45b558 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:00.294461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54df4-6456-4cc8-bf9a-5ef7fa45b558 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:00.294563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad54df4-6456-4cc8-bf9a-5ef7fa45b558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:00.312641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2ad54df4-6456-4cc8-bf9a-5ef7fa45b558 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:00.313838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2ad54df4-6456-4cc8-bf9a-5ef7fa45b558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:00.393987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 631/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:09:00.419971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fcc2a-7dfc-4580-b164-c166183be84b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:00.448540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:00.537888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fcc2a-7dfc-4580-b164-c166183be84b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:00.579179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:00.579939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:00.580270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:00.580604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:00.593868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:09:00.603649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:00.605525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:00.607158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 604/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Jun 24 09:09:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:00.611539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad54df4-6456-4cc8-bf9a-5ef7fa45b558 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:00.678689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:00.761451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad54df4-6456-4cc8-bf9a-5ef7fa45b558 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:00.807788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:00.808929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:00.809653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:00.810128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:00.811501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:00.812038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:00.814599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:00.829569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 632/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Jun 24 09:09:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:09:00.836846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:00.968027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:00.986342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74aae3-19d4-47ca-af7b-1489c5ea7d7e None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:01.034090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:01.035173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:01.035632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:01.036125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:01.051386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:01.052878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:01.071239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:01.073763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:01.076211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:01.076211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:01.076211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:01.076211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:01.076211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 605/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:01.081617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:01.224304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6e647-f94a-4c02-8b6a-4d45232b11f2 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:01.246598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:01.286592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:01.287628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:01.288095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:01.288613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:01.289813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:01.290368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:01.291906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:01.304958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 633/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:01.311361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:01.452084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b000c-de46-4420-be8f-52bbaad352e1 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:01.496521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:01.497806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:01.498233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:01.498666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:01.502795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:01.512057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:01.514656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:09:01.531934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:01.533847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:01.535701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 606/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:01.540835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:01.659489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752ca84-ec88-4596-b1a3-9511cd524e7b None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:01.699660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:01.700433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:01.700791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:01.701007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:01.701940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:01.702364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:01.703781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:01.725296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 634/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Jun 24 09:09:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:01.733109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:01.862640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8f849-cd7c-4139-b8f2-e88c61ad1936 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:01.913428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:01.914084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:01.914367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:01.914646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:01.989938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:01.997807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:01.999247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:02.119735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 607/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:09:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:09:02.126106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:02.159578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:02.260250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a7410-097e-4669-8161-788a2b8b2d97 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:02.299114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:02.300105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:02.300462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:02.300816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:02.313839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:02.315620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:02.339770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:02.341023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:02.342057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:02.342057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:02.342057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:02.342057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:02.348364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:02.349189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 635/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:02.459978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:02.466621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac0d01-3929-4720-9ecf-516070475db6 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:02.506258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:02.506959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:02.507242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:02.507536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:02.508247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:02.508591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:02.510590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:02.522742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 608/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:02.528704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:02.615867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:02.640156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0982a7-dca8-400a-a727-ac331af54a2a None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:02.679608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:02.680469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:02.680860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:02.681934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:02.690829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:02.692407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:09:02.707973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:02.709101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:02.714166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 636/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:09:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:09:02.716581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:02.827951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233c241-4f09-4160-b446-c66831b35732 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:02.869232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a1de-ee6f-4021-ba78-425f074fd82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:02.870426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a1de-ee6f-4021-ba78-425f074fd82e None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:02.870904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a1de-ee6f-4021-ba78-425f074fd82e None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:02.871463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a1de-ee6f-4021-ba78-425f074fd82e None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:02.872360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a1de-ee6f-4021-ba78-425f074fd82e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:02.872911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a1de-ee6f-4021-ba78-425f074fd82e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:02.874770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a1de-ee6f-4021-ba78-425f074fd82e None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:02.894162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 609/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:09:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:02.900874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:03.044705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b95330-faa9-4aaa-9760-90ca91700631 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:03.092645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90cea4-20aa-4058-a77f-c0dab96df011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:03.093546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90cea4-20aa-4058-a77f-c0dab96df011 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:03.093808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90cea4-20aa-4058-a77f-c0dab96df011 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:03.094248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90cea4-20aa-4058-a77f-c0dab96df011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:03.095344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90cea4-20aa-4058-a77f-c0dab96df011 None admin] RBAC: Adding query filter params (role.id=9654f3986243401db4f1aad536273ede, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=505dda787dac43bca3595fa1c406a457) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:03.095739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90cea4-20aa-4058-a77f-c0dab96df011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:03.097654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90cea4-20aa-4058-a77f-c0dab96df011 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:03.105932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 637/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:09:02 2022] GET /identity/v3/role_assignments?user.id=505dda787dac43bca3595fa1c406a457&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=9654f3986243401db4f1aad536273ede => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:05.405939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b96ba4-2d36-4d84-8f76-b48ab8d451e7 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:05.406461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b96ba4-2d36-4d84-8f76-b48ab8d451e7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:05.406825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b96ba4-2d36-4d84-8f76-b48ab8d451e7 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:05.408985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 610/1247] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:09:05.416363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0094b-a954-4f09-9589-ba942b3e044b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:05.416901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0094b-a954-4f09-9589-ba942b3e044b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:05.417235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0094b-a954-4f09-9589-ba942b3e044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:05.424631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:05.440322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-38c0094b-a954-4f09-9589-ba942b3e044b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:05.441578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-38c0094b-a954-4f09-9589-ba942b3e044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:05.527555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 638/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:09:05.553382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b96ba4-2d36-4d84-8f76-b48ab8d451e7 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:05.576530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:05.690632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b96ba4-2d36-4d84-8f76-b48ab8d451e7 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:05.735104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:05.735942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:05.736225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:05.736502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:05.748769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:09:05.756764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:05.757756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:05.758818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 611/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:09:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:05.764937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0094b-a954-4f09-9589-ba942b3e044b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:05.910974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0094b-a954-4f09-9589-ba942b3e044b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:05.949599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:05.956137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:05.956849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:05.957154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:05.957407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:05.958439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:05.958795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:05.960452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:05.971559 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 639/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:09:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:05.979500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:06.098957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:06.134176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df823b8-42c4-4f6d-a2a9-38d780b4e0cc None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:06.182515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:06.183528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:06.183816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:06.184362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:06.199495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:06.201075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:06.222639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:06.225969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:06.228582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:06.228582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:06.228582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:06.228582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:06.228582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 612/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:09:06.230938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:06.365590 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797080f-1c3f-4fbd-930c-3c8e287c2710 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:06.408098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:06.409167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:06.409657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:06.410151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:06.411480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:06.412014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:06.414514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:06.431929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 640/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:09:06.440501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:06.574185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6772c2-6b9a-4fc5-9527-7c441306120c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:06.605322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:06.615425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:06.616243 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:06.616530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:06.616839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:06.625793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:06.627473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:09:06.645129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:06.646551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:06.649001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 613/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:09:06.654455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:06.720680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:06.775176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc0c1-79ba-4092-84df-e196e57e08b5 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:06.815438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:06.816119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:06.816402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:06.816712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:06.817671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:06.818008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:06.819600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:06.849092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 641/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Jun 24 09:09:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:06.850769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:06.988292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19132598-f7b3-4828-85a3-d51778e0186f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:07.030703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:07.031492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:07.031939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:07.032358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:07.113506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:07.135017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:07.136665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:07.270625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 614/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:09:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:09:07.278417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:07.405315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028a674-5235-4fbd-9ec4-b1c544012675 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:07.421232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:07.447447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:07.448745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:07.449117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:07.449523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:07.465532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:07.467697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:07.491919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:07.493578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:07.494522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:07.494522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:07.494522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:07.494522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:07.494522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 642/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:07.499804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:07.604018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:07.617645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb9f4a-9782-4a97-bfe9-bcfd1660cd1c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:07.661116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:07.661683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:07.661983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:07.662372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:07.663299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:07.663611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:07.665384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:07.679558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 615/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:07.686789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:07.812137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0aaef-c60a-4048-9680-70ca2cfcbfec None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:07.871302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:07.872665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:07.873155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:07.873637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:07.882287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:07.883799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:09:07.900000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:07.901147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:07.902305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:07.908281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 643/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:07.908409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:08.039291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b75c6-9a99-4987-9738-70f136dce789 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:08.087164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef41b09-ba7a-4ac8-8b6a-aba621161dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:08.088128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41b09-ba7a-4ac8-8b6a-aba621161dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:08.088665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41b09-ba7a-4ac8-8b6a-aba621161dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:08.089211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41b09-ba7a-4ac8-8b6a-aba621161dd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:08.090608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41b09-ba7a-4ac8-8b6a-aba621161dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:08.091222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41b09-ba7a-4ac8-8b6a-aba621161dd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:08.093496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41b09-ba7a-4ac8-8b6a-aba621161dd3 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:08.109144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 616/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:08.115778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:08.150128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:08.246632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7be00-4d08-4daa-bdf5-298720ef683e None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:08.295204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608beab-8df3-4abb-ae22-57c2cc301958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:08.295831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608beab-8df3-4abb-ae22-57c2cc301958 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:08.296121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608beab-8df3-4abb-ae22-57c2cc301958 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:08.296404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608beab-8df3-4abb-ae22-57c2cc301958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:08.297421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608beab-8df3-4abb-ae22-57c2cc301958 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=505dda787dac43bca3595fa1c406a457) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:08.297814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608beab-8df3-4abb-ae22-57c2cc301958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:08.299451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608beab-8df3-4abb-ae22-57c2cc301958 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:08.306897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 644/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:09:08 2022] GET /identity/v3/role_assignments?user.id=505dda787dac43bca3595fa1c406a457&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:09:10.382261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd6a4-453e-4fc9-af41-5f1dbfc3a57c None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:10.382908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd6a4-453e-4fc9-af41-5f1dbfc3a57c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:10.382908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edd6a4-453e-4fc9-af41-5f1dbfc3a57c None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:10.384100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 617/1261] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:09:10.389493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597843a-4255-43bb-82a1-77cda2e648c8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:10.389945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597843a-4255-43bb-82a1-77cda2e648c8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:10.390058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597843a-4255-43bb-82a1-77cda2e648c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:10.407144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c597843a-4255-43bb-82a1-77cda2e648c8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:10.408331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c597843a-4255-43bb-82a1-77cda2e648c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:10.474339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 645/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:09:10.497605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edd6a4-453e-4fc9-af41-5f1dbfc3a57c None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:10.508491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:10.599469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edd6a4-453e-4fc9-af41-5f1dbfc3a57c None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:10.634567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:10.635265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:10.635543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:10.635814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:10.645729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:09:10.653886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:10.655088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:10.656024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 618/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:09:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:10.661515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597843a-4255-43bb-82a1-77cda2e648c8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:10.679809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:10.756962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597843a-4255-43bb-82a1-77cda2e648c8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:10.789926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:10.790604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:10.790881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:10.791159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:10.791891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:10.792189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:10.793571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:10.801403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 646/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:09:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:10.807233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:10.891809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:10.929097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd7570-2bf7-4d79-8d45-ab20b5af3c72 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:10.964111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:10.964971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:10.965257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:10.965534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:10.976349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:10.977874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:10.995202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:10.996383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:10.997725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:10.997725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:10.997725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:10.997725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:10.997725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 619/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:11.002930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:11.096076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:11.115132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaac009-9534-47b5-b09a-ba8226cb31ce None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:11.148363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:11.148972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:11.149254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:11.150107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:11.150846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:11.151147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:11.152501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:11.164206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 647/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:11.170048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:11.269930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2001f03-10a8-4bd3-aea8-2eb213a2c43c None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:11.288790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:11.303244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:11.304050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:11.304368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:11.304686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:11.313209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:11.314649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:09:11.331762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:11.332979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:11.333990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 620/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:11.339758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:11.442542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1ef7-9581-4a8e-90c0-b702e9e2c478 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:11.475806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:11.476469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:11.476780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:11.477055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:11.477930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:11.478235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:11.479654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:11.488289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:11.498948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 648/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Jun 24 09:09:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:11.504729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:11.614019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a7479-165c-4659-8b93-b7133f7a0e0f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:11.648752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:11.649580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:11.649867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:11.650147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:11.660945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:11.662393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:11.679840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:11.681023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:11.681948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:11.681948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:11.681948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:11.681948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:11.681948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 621/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:11.687681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:11.785429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0729-f306-4290-8484-621ba0065ca7 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:11.822968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:11.823689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:11.824028 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:11.824353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:11.825447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:11.825927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:11.828682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:11.848207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 649/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:11.857361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:11.890353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:11.958776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7d435-94c3-4308-904f-8e3f5b319459 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:11.992434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:11.993261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:11.993537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:11.993807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:12.000384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:12.001846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:09:12.015629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:12.016736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:12.017709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 622/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:09:12.025566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:12.085453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:12.127992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad13a9-d5e9-4c99-b24c-71b15a70e0cf None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:12.164014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262af8-98f7-40a2-b76f-b710f0977553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:12.164698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262af8-98f7-40a2-b76f-b710f0977553 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:12.164998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262af8-98f7-40a2-b76f-b710f0977553 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:12.165823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262af8-98f7-40a2-b76f-b710f0977553 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:12.166599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262af8-98f7-40a2-b76f-b710f0977553 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:12.166902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262af8-98f7-40a2-b76f-b710f0977553 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:12.168321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262af8-98f7-40a2-b76f-b710f0977553 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:12.180033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 650/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:09:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:12.186491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:12.277382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:12.287071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c7879-74e5-484d-bc93-848bf58fb9ce None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:12.320092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d1427-c7f5-4909-aeb3-aba84c3b7584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:12.321017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d1427-c7f5-4909-aeb3-aba84c3b7584 None admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:12.321801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d1427-c7f5-4909-aeb3-aba84c3b7584 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:12.322077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d1427-c7f5-4909-aeb3-aba84c3b7584 None admin] PATH_INFO: `/v3/projects/f199023babeb4ffa88076256ba736ba1/users/505dda787dac43bca3595fa1c406a457/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:12.324009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d1427-c7f5-4909-aeb3-aba84c3b7584 None admin] RBAC: Authorizing `identity:create_grant(project_id=f199023babeb4ffa88076256ba736ba1, user_id=505dda787dac43bca3595fa1c406a457, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:12.325466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d1427-c7f5-4909-aeb3-aba84c3b7584 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:12.339024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 623/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Jun 24 09:09:12 2022] PUT /identity/v3/projects/f199023babeb4ffa88076256ba736ba1/users/505dda787dac43bca3595fa1c406a457/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:09:14.388315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893ad0e-65f4-4f68-ad80-db038ce7a850 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:14.388916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893ad0e-65f4-4f68-ad80-db038ce7a850 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:14.388916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893ad0e-65f4-4f68-ad80-db038ce7a850 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:14.390179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 651/1274] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:09:14.395374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9a9e7-1b29-4c84-87b7-11fb47505427 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:14.395814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9a9e7-1b29-4c84-87b7-11fb47505427 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:14.395909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9a9e7-1b29-4c84-87b7-11fb47505427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:14.412169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-18d9a9e7-1b29-4c84-87b7-11fb47505427 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:14.413357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-18d9a9e7-1b29-4c84-87b7-11fb47505427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:14.478903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 624/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:09:14.502826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893ad0e-65f4-4f68-ad80-db038ce7a850 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:14.604700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893ad0e-65f4-4f68-ad80-db038ce7a850 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:14.622506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:14.640094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:14.640845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:14.641217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:14.641500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:14.656672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:09:14.664453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:14.666203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:14.667337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 652/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Jun 24 09:09:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:14.672513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9a9e7-1b29-4c84-87b7-11fb47505427 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:14.753406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:14.771676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9a9e7-1b29-4c84-87b7-11fb47505427 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:14.805149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:14.806367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:14.806646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:14.806916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:14.807649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:14.807949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:14.809584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:14.818111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 625/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Jun 24 09:09:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:14.824233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:14.938026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e6d9-5191-4b86-a2de-7cd86807f9ef None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:14.971866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:14.972697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:14.972978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:14.973249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:14.986571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:14.988022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:15.006591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:15.007760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:15.009042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:15.009042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:15.009042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:15.009042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:15.009042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 653/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:15.013690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:15.113927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f783e6-b53f-44a9-b7f8-79addf16df06 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:15.146340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:15.146996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:15.147271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:15.147541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:15.148259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:15.148573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:15.150607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:15.163277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 626/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:15.169429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:15.199357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:15.271129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b4797-5818-40f3-b86d-b6e498c60680 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:15.305590 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:15.306372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:15.306656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:15.306929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:15.315464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:15.316932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:09:15.332667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:15.333714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:15.334839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 654/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:15.340586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:15.344822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:15.440157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edf3b4-59e2-42eb-bd28-c88f1ff854a7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:15.473726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:15.474316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:15.474597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:15.474867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:15.475690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:15.475994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:15.477475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:15.494759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 627/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Jun 24 09:09:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:15.500670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:15.591735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:15.602138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1151b5-f5fc-4a76-8579-e5b57ee44b92 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:15.635425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:15.636067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:15.636347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:15.636652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:15.705294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:15.706741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:15.794743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:15.809451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 655/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Jun 24 09:09:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:09:15.815724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:15.878297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:15.917284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b153-556a-4c01-83b0-dba2ea96a69f None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:15.952702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:15.953493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:15.953777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:15.954053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:15.964079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:15.965550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:09:15.982789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:15.983841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:15.984829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:09:15.984829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:09:15.984829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:09:15.984829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:15.984829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 628/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:09:15.989704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:16.091584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecaffc-f21c-4910-8b1b-773ecbb8d54d None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:16.125695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:16.126355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:16.126640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:16.126919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:16.127641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:16.127946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:16.129347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:16.140603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 656/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Jun 24 09:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:16.146521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:16.250544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a3587-0f63-4854-963b-5a33e2ffd2c7 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:16.264593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:16.284387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:16.285207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:16.285480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:16.285750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:16.292863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:16.294295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:09:16.308381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:16.309840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: ERROR keystone.server.flask.application Jun 24 09:09:16.311062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 629/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Jun 24 09:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:16.315801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:16.371302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:16.426038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d73e9c-ce44-4606-9e3c-140072381d28 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:16.462895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf5937-0f47-4237-98e6-9df627db1f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:16.463551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf5937-0f47-4237-98e6-9df627db1f97 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:16.463833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf5937-0f47-4237-98e6-9df627db1f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:16.464110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf5937-0f47-4237-98e6-9df627db1f97 None admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:16.464922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf5937-0f47-4237-98e6-9df627db1f97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:16.465229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf5937-0f47-4237-98e6-9df627db1f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:16.466658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf5937-0f47-4237-98e6-9df627db1f97 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:16.481345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 657/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Jun 24 09:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:09:16.488002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:16.585529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab23422-e1f3-4137-ac9d-65c7921cac44 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:16.620499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3128be3-857e-4869-80cc-1cc964a133a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:16.621173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3128be3-857e-4869-80cc-1cc964a133a5 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:16.621504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3128be3-857e-4869-80cc-1cc964a133a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:16.621724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3128be3-857e-4869-80cc-1cc964a133a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:16.622538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3128be3-857e-4869-80cc-1cc964a133a5 None admin] RBAC: Adding query filter params (role.id=40cc7f5bdd5e4702924b038b7f946938, scope.project.id=f199023babeb4ffa88076256ba736ba1, user.id=505dda787dac43bca3595fa1c406a457) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:16.622842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3128be3-857e-4869-80cc-1cc964a133a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:16.624259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3128be3-857e-4869-80cc-1cc964a133a5 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:16.630156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 630/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Jun 24 09:09:16 2022] GET /identity/v3/role_assignments?user.id=505dda787dac43bca3595fa1c406a457&scope.project.id=f199023babeb4ffa88076256ba736ba1&role.id=40cc7f5bdd5e4702924b038b7f946938 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:18.711207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d652fa-59f9-4075-8d8e-fa8e330e700c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:18.711826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d652fa-59f9-4075-8d8e-fa8e330e700c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:18.711826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d652fa-59f9-4075-8d8e-fa8e330e700c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:18.713174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 658/1288] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:09:18.718556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d955741-36c8-4280-81fa-5f7b99335753 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:18.719033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d955741-36c8-4280-81fa-5f7b99335753 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:18.719144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d955741-36c8-4280-81fa-5f7b99335753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:18.737688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-2d955741-36c8-4280-81fa-5f7b99335753 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:18.738857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2d955741-36c8-4280-81fa-5f7b99335753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:18.774719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:18.815982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 631/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:09:18.842690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d652fa-59f9-4075-8d8e-fa8e330e700c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:18.959378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d652fa-59f9-4075-8d8e-fa8e330e700c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:18.970578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:18.994856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:18.995656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:18.995935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:18.996204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:18.996889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:18.998446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:19.013258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:19.014533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:19.015399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 659/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:09:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:09:19.020786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d955741-36c8-4280-81fa-5f7b99335753 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:19.123474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d955741-36c8-4280-81fa-5f7b99335753 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:19.157699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e770-29a1-4af8-b6ce-aa2a511a5760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:19.158375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e770-29a1-4af8-b6ce-aa2a511a5760 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:19.158657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e770-29a1-4af8-b6ce-aa2a511a5760 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:19.158932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e770-29a1-4af8-b6ce-aa2a511a5760 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:19.159645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e770-29a1-4af8-b6ce-aa2a511a5760 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:19.159948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e770-29a1-4af8-b6ce-aa2a511a5760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:19.161341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e770-29a1-4af8-b6ce-aa2a511a5760 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:19.166932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 632/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Jun 24 09:09:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:19.172797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:19.276596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d37810-830b-494a-b110-d6ea10914148 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:19.315112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31210b8d-3a1e-4537-a444-4e415d94e86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:19.315780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31210b8d-3a1e-4537-a444-4e415d94e86b None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:19.316063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31210b8d-3a1e-4537-a444-4e415d94e86b None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:19.316340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31210b8d-3a1e-4537-a444-4e415d94e86b None admin] PATH_INFO: `/v3/services` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:19.317082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31210b8d-3a1e-4537-a444-4e415d94e86b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:19.317413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31210b8d-3a1e-4537-a444-4e415d94e86b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:19.318790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31210b8d-3a1e-4537-a444-4e415d94e86b None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:19.323348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 660/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Jun 24 09:09:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:21.335614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8e21-43ae-4a00-8469-9ded91227a5a None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:21.336114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8e21-43ae-4a00-8469-9ded91227a5a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:21.336114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe8e21-43ae-4a00-8469-9ded91227a5a None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:21.338205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 633/1293] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:09:21.343231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22175234-03d6-4ac4-8bdf-71c006eb467a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:21.343669 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22175234-03d6-4ac4-8bdf-71c006eb467a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:21.343773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22175234-03d6-4ac4-8bdf-71c006eb467a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:21.360115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-22175234-03d6-4ac4-8bdf-71c006eb467a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:21.361327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-22175234-03d6-4ac4-8bdf-71c006eb467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:21.389932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:21.426286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 661/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:09:21.449110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe8e21-43ae-4a00-8469-9ded91227a5a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:21.479540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:21.558719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe8e21-43ae-4a00-8469-9ded91227a5a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:21.592235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbc056-cf80-49fd-9eb1-e682a3bd7a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:21.592859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc056-cf80-49fd-9eb1-e682a3bd7a46 None admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:21.593140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc056-cf80-49fd-9eb1-e682a3bd7a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:21.593410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbc056-cf80-49fd-9eb1-e682a3bd7a46 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:21.594111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbc056-cf80-49fd-9eb1-e682a3bd7a46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:21.595469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbc056-cf80-49fd-9eb1-e682a3bd7a46 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:21.608316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 634/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Jun 24 09:09:21 2022] POST /identity/v3/services => generated 246 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:09:23.610989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037495c1-f8b4-4f45-9605-81b492e5164e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:23.611518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037495c1-f8b4-4f45-9605-81b492e5164e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:23.611518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037495c1-f8b4-4f45-9605-81b492e5164e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:23.612938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 662/1296] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:09:23.618861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825f19-51fe-4806-997a-cf5f9901b969 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:23.619294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825f19-51fe-4806-997a-cf5f9901b969 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:23.619388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825f19-51fe-4806-997a-cf5f9901b969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:23.635593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-50825f19-51fe-4806-997a-cf5f9901b969 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:23.636787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-50825f19-51fe-4806-997a-cf5f9901b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:23.706955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 635/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:09:23.730395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037495c1-f8b4-4f45-9605-81b492e5164e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:23.831975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037495c1-f8b4-4f45-9605-81b492e5164e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:23.867609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:23.868953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:23.869240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:23.869520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] PATH_INFO: `/v3/services/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:23.870173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:23.871537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:23.883136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:23.884334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:23.885660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 663/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:09:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:23.890311 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825f19-51fe-4806-997a-cf5f9901b969 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:23.908404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:23.987791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825f19-51fe-4806-997a-cf5f9901b969 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:24.021449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:24.022113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:24.022395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:24.022666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:24.023373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:24.023677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:24.025578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:24.030973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 636/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Jun 24 09:09:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:24.036772 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:24.070288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:24.152788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c747dc7-ec9b-47f1-a485-8d1c1ddc9aab None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:24.193351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321fd70-4592-4acb-b5c1-2086084f0343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:24.194015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321fd70-4592-4acb-b5c1-2086084f0343 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:24.194294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321fd70-4592-4acb-b5c1-2086084f0343 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:24.194566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321fd70-4592-4acb-b5c1-2086084f0343 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:24.195354 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321fd70-4592-4acb-b5c1-2086084f0343 None admin] RBAC: Adding query filter params (interface=public, service_id=d9fae4b44f1041788044d095d6dd0cd2, region_id=RegionOne) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:24.195662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321fd70-4592-4acb-b5c1-2086084f0343 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:24.197058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321fd70-4592-4acb-b5c1-2086084f0343 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:24.202108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 664/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Jun 24 09:09:24 2022] GET /identity/v3/endpoints?service_id=d9fae4b44f1041788044d095d6dd0cd2&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:24.207757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:24.296866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:24.317166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067925d6-a2c2-4905-a815-06a0ca8afd45 None admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:24.350131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a2763-8e11-4ff0-ac3f-a64d1298b88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:24.350788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a2763-8e11-4ff0-ac3f-a64d1298b88f None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:24.351065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a2763-8e11-4ff0-ac3f-a64d1298b88f None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:24.351337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a2763-8e11-4ff0-ac3f-a64d1298b88f None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:24.352007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a2763-8e11-4ff0-ac3f-a64d1298b88f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:24.353999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a2763-8e11-4ff0-ac3f-a64d1298b88f None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:24.360907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 637/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Jun 24 09:09:24 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:09:26.382700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023309-7b04-43fe-b7ef-138feacd8a93 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:26.383207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023309-7b04-43fe-b7ef-138feacd8a93 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:26.383207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023309-7b04-43fe-b7ef-138feacd8a93 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:26.385331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 665/1302] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:09:26.390861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93811b73-9b8c-48df-9f77-af5eb31814aa None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:26.391284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93811b73-9b8c-48df-9f77-af5eb31814aa None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:26.391414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93811b73-9b8c-48df-9f77-af5eb31814aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:26.408107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-93811b73-9b8c-48df-9f77-af5eb31814aa None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:09:26.409328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-93811b73-9b8c-48df-9f77-af5eb31814aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:09:26.474152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 638/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:09:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:09:26.497495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6023309-7b04-43fe-b7ef-138feacd8a93 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:26.591657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:26.598783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6023309-7b04-43fe-b7ef-138feacd8a93 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:26.632478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:26.633352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:26.633617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:26.633897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:26.634543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:26.635916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:09:26.647639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:09:26.648863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application with Lock( Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: ERROR keystone.server.flask.application Jun 24 09:09:26.650037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 666/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Jun 24 09:09:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:09:26.654897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93811b73-9b8c-48df-9f77-af5eb31814aa None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:26.731660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:26.755591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93811b73-9b8c-48df-9f77-af5eb31814aa None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:26.790384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b1969-1e63-427a-90f7-afc322f3df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:26.791087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1969-1e63-427a-90f7-afc322f3df41 None admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:26.791359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1969-1e63-427a-90f7-afc322f3df41 None admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:26.791636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1969-1e63-427a-90f7-afc322f3df41 None admin] PATH_INFO: `/v3/services` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:26.792342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b1969-1e63-427a-90f7-afc322f3df41 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:09:26.792673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b1969-1e63-427a-90f7-afc322f3df41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:26.794042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b1969-1e63-427a-90f7-afc322f3df41 None admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:26.799377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 639/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Jun 24 09:09:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:09:26.805125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:09:26.908205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c38b3-b41f-4bbb-a639-6c1e419c04f4 None admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:09:26.942846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767705a0-9b5a-4eec-9f99-1def59b08fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:09:26.943507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767705a0-9b5a-4eec-9f99-1def59b08fa4 None admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:09:26.943787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767705a0-9b5a-4eec-9f99-1def59b08fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:09:26.944059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767705a0-9b5a-4eec-9f99-1def59b08fa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:09:26.944831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767705a0-9b5a-4eec-9f99-1def59b08fa4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:09:26.946248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767705a0-9b5a-4eec-9f99-1def59b08fa4 None admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:09:26.965141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 667/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Jun 24 09:09:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:09:36.796261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=30 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:09:36.960422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:31.116320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f171d-94a7-4dbc-a75e-dadd7b7b589c None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:31.117555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f171d-94a7-4dbc-a75e-dadd7b7b589c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:31.118064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f171d-94a7-4dbc-a75e-dadd7b7b589c None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:31.120684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 640/1307] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:31.126510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f257b-d873-4df9-8e70-3ae3b46dfa2e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:31.127095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f257b-d873-4df9-8e70-3ae3b46dfa2e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:31.127095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f257b-d873-4df9-8e70-3ae3b46dfa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:31.209846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e91f257b-d873-4df9-8e70-3ae3b46dfa2e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:31.211898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e91f257b-d873-4df9-8e70-3ae3b46dfa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:31.348396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 668/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:31.399223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a454ab-0b34-4f6d-915a-652efcafc70a None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:31.399944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a454ab-0b34-4f6d-915a-652efcafc70a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:31.400619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a454ab-0b34-4f6d-915a-652efcafc70a None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:31.402936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 641/1309] 10.222.0.52 () {54 vars in 964 bytes} [Fri Jun 24 09:10:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:31.409334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87247db-64fa-48f3-b08c-3a1f590b4ec2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:31.409628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87247db-64fa-48f3-b08c-3a1f590b4ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:31.409931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87247db-64fa-48f3-b08c-3a1f590b4ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:31.500877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d87247db-64fa-48f3-b08c-3a1f590b4ec2 None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:31.504101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d87247db-64fa-48f3-b08c-3a1f590b4ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:31.555104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:31.640637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 669/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Fri Jun 24 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:31.650361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a454ab-0b34-4f6d-915a-652efcafc70a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:31.652415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-65a454ab-0b34-4f6d-915a-652efcafc70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:31.802453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a454ab-0b34-4f6d-915a-652efcafc70a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:31.887309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ad274-0930-4704-97cd-93796918eb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:31.887962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ad274-0930-4704-97cd-93796918eb88 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:31.888237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ad274-0930-4704-97cd-93796918eb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:31.888514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ad274-0930-4704-97cd-93796918eb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:31.890196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-6a3ad274-0930-4704-97cd-93796918eb88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:31.947880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:32.011303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ad274-0930-4704-97cd-93796918eb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:32.012736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ad274-0930-4704-97cd-93796918eb88 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:32.032880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 642/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:34.443373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f475e-91f2-4e83-8c2e-e240b2837acb None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:34.443898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f475e-91f2-4e83-8c2e-e240b2837acb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:34.443898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f475e-91f2-4e83-8c2e-e240b2837acb None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:34.445900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 670/1312] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:34.450942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9920-e046-4960-b5d1-00308644bc0c None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:34.451358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9920-e046-4960-b5d1-00308644bc0c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:34.451551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9920-e046-4960-b5d1-00308644bc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:34.543910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e09d9920-e046-4960-b5d1-00308644bc0c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:34.545476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e09d9920-e046-4960-b5d1-00308644bc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:34.653230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 643/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:34.691245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f475e-91f2-4e83-8c2e-e240b2837acb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:34.692856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b39f475e-91f2-4e83-8c2e-e240b2837acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:34.817048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f475e-91f2-4e83-8c2e-e240b2837acb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:34.905236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:34.909363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ecaea-4760-4425-95a0-2c60ba197f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:34.910668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ecaea-4760-4425-95a0-2c60ba197f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:34.911151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ecaea-4760-4425-95a0-2c60ba197f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:34.911438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ecaea-4760-4425-95a0-2c60ba197f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:34.913968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b43ecaea-4760-4425-95a0-2c60ba197f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:35.036297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ecaea-4760-4425-95a0-2c60ba197f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:35.038199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ecaea-4760-4425-95a0-2c60ba197f96 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:35.056787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 671/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:37.512868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3624682-a3fb-42af-af93-917561d93380 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:37.513375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3624682-a3fb-42af-af93-917561d93380 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:37.513375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3624682-a3fb-42af-af93-917561d93380 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:37.514719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 644/1315] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:37.519788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7721-4617-4604-a64e-19d965d88f4c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:37.520229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7721-4617-4604-a64e-19d965d88f4c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:37.520334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7721-4617-4604-a64e-19d965d88f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:37.586309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5aea7721-4617-4604-a64e-19d965d88f4c None None] MFA Rules not processed for user `c777cddf95b3464587cd81664f9af1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:37.587523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5aea7721-4617-4604-a64e-19d965d88f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:37.638781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:37.685929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 672/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:37.725493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3624682-a3fb-42af-af93-917561d93380 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:37.727913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b3624682-a3fb-42af-af93-917561d93380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:37.737573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:37.833402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3624682-a3fb-42af-af93-917561d93380 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:37.908074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6bee9-b2d2-42b6-8126-86e43583a943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:37.908814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bee9-b2d2-42b6-8126-86e43583a943 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:37.909098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bee9-b2d2-42b6-8126-86e43583a943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:37.909370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bee9-b2d2-42b6-8126-86e43583a943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:37.911030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-46f6bee9-b2d2-42b6-8126-86e43583a943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:37.975293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:38.023855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6bee9-b2d2-42b6-8126-86e43583a943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:38.025328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6bee9-b2d2-42b6-8126-86e43583a943 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:38.043803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 645/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:40.686169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21862-e87f-4deb-b9c9-20a529c78ade None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:40.686895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21862-e87f-4deb-b9c9-20a529c78ade None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:40.686895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21862-e87f-4deb-b9c9-20a529c78ade None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:40.688215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 673/1318] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:40.699973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4702493-006d-41d5-ac89-74309830bd77 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:40.700638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4702493-006d-41d5-ac89-74309830bd77 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:40.700638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4702493-006d-41d5-ac89-74309830bd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:40.790247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-b4702493-006d-41d5-ac89-74309830bd77 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:40.791955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b4702493-006d-41d5-ac89-74309830bd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:40.920627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 646/1319] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:40.957776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c21862-e87f-4deb-b9c9-20a529c78ade None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:40.962145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e7c21862-e87f-4deb-b9c9-20a529c78ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:41.075061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c21862-e87f-4deb-b9c9-20a529c78ade None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:41.151356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec4b40-21e2-48ae-b693-adf0c8367370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:41.151992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec4b40-21e2-48ae-b693-adf0c8367370 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:41.152269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec4b40-21e2-48ae-b693-adf0c8367370 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:41.152536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec4b40-21e2-48ae-b693-adf0c8367370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:41.154223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4eec4b40-21e2-48ae-b693-adf0c8367370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:41.163852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:41.275426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec4b40-21e2-48ae-b693-adf0c8367370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:41.276888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec4b40-21e2-48ae-b693-adf0c8367370 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:41.295325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 674/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:44.104814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb9636-fac8-4bb4-bacf-8b8d1d6da1e9 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:44.105499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb9636-fac8-4bb4-bacf-8b8d1d6da1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:44.105499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb9636-fac8-4bb4-bacf-8b8d1d6da1e9 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:44.106765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 647/1321] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:44.113279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd627d-6597-4ee0-841e-f0a87ff26ab6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:44.113857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd627d-6597-4ee0-841e-f0a87ff26ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:44.113857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd627d-6597-4ee0-841e-f0a87ff26ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:44.187782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a4fd627d-6597-4ee0-841e-f0a87ff26ab6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:44.189042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4fd627d-6597-4ee0-841e-f0a87ff26ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:44.259395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:44.319060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 675/1322] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:44.372426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38635c68-cc17-4547-9cfb-fb969ec319f2 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:44.372751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38635c68-cc17-4547-9cfb-fb969ec319f2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:44.373044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38635c68-cc17-4547-9cfb-fb969ec319f2 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:44.374459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 648/1323] 10.222.0.52 () {54 vars in 964 bytes} [Fri Jun 24 09:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:10:44.381246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08047662-1b37-43b4-80a8-284c12a06269 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:44.381569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08047662-1b37-43b4-80a8-284c12a06269 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:44.381883 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08047662-1b37-43b4-80a8-284c12a06269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:44.470218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-08047662-1b37-43b4-80a8-284c12a06269 None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:44.471417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-08047662-1b37-43b4-80a8-284c12a06269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:44.555063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 676/1324] 10.222.0.52 () {58 vars in 1061 bytes} [Fri Jun 24 09:10:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:44.564705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38635c68-cc17-4547-9cfb-fb969ec319f2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:44.566405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-38635c68-cc17-4547-9cfb-fb969ec319f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:44.592647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:44.685138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38635c68-cc17-4547-9cfb-fb969ec319f2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:44.764612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a89861-3382-40db-ae23-36be5dbac590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:44.765280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a89861-3382-40db-ae23-36be5dbac590 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:44.765564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a89861-3382-40db-ae23-36be5dbac590 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:44.765837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a89861-3382-40db-ae23-36be5dbac590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:44.767536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-43a89861-3382-40db-ae23-36be5dbac590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:44.854983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:44.897211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a89861-3382-40db-ae23-36be5dbac590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:44.899156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a89861-3382-40db-ae23-36be5dbac590 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:44.918896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 649/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:10:47.291266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f7b99-31a7-4ca7-aa7d-1e9184ad3176 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:47.291777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f7b99-31a7-4ca7-aa7d-1e9184ad3176 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:47.291777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f7b99-31a7-4ca7-aa7d-1e9184ad3176 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:47.293181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 677/1326] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:47.298682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6abb52-c801-4ee9-bf36-cd79b36e12a2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:47.299149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6abb52-c801-4ee9-bf36-cd79b36e12a2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:47.299242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6abb52-c801-4ee9-bf36-cd79b36e12a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:47.370436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-dc6abb52-c801-4ee9-bf36-cd79b36e12a2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:47.371662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-dc6abb52-c801-4ee9-bf36-cd79b36e12a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:47.489540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 650/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:47.525950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f7b99-31a7-4ca7-aa7d-1e9184ad3176 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:47.527576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-556f7b99-31a7-4ca7-aa7d-1e9184ad3176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:47.595333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:47.652997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f7b99-31a7-4ca7-aa7d-1e9184ad3176 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:47.740267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e227df1-cb79-431b-a3ac-4ce655b147c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:47.740874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227df1-cb79-431b-a3ac-4ce655b147c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:47.741680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227df1-cb79-431b-a3ac-4ce655b147c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:47.741952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227df1-cb79-431b-a3ac-4ce655b147c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:47.743627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7e227df1-cb79-431b-a3ac-4ce655b147c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:47.857118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:47.859742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e227df1-cb79-431b-a3ac-4ce655b147c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:47.861767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e227df1-cb79-431b-a3ac-4ce655b147c5 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:47.881631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 678/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:50.055956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e6dfc-5142-4e58-9db2-836048b34b77 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:50.056397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e6dfc-5142-4e58-9db2-836048b34b77 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:50.056502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e6dfc-5142-4e58-9db2-836048b34b77 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:50.059267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 651/1329] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:50.067334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7deda6-0e1a-4627-924a-b37614be054f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:50.067334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7deda6-0e1a-4627-924a-b37614be054f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:50.067334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7deda6-0e1a-4627-924a-b37614be054f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:50.150696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8d7deda6-0e1a-4627-924a-b37614be054f None None] MFA Rules not processed for user `c777cddf95b3464587cd81664f9af1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:50.151925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8d7deda6-0e1a-4627-924a-b37614be054f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:50.255549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 679/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:50.293319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e6dfc-5142-4e58-9db2-836048b34b77 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:50.294872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-628e6dfc-5142-4e58-9db2-836048b34b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:50.327219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:50.406116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e6dfc-5142-4e58-9db2-836048b34b77 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:50.484292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4662c-eaa3-48cd-9a1f-57e1ea2f2ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:50.485053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4662c-eaa3-48cd-9a1f-57e1ea2f2ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:50.485376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4662c-eaa3-48cd-9a1f-57e1ea2f2ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:50.485657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4662c-eaa3-48cd-9a1f-57e1ea2f2ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:50.487370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3ae4662c-eaa3-48cd-9a1f-57e1ea2f2ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:50.569841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:50.602558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4662c-eaa3-48cd-9a1f-57e1ea2f2ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:50.604007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4662c-eaa3-48cd-9a1f-57e1ea2f2ce2 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:50.622988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 652/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:53.135075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde403e-e3c3-4ae7-8015-d0a38b9f0eb4 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:53.135598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde403e-e3c3-4ae7-8015-d0a38b9f0eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:53.135694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde403e-e3c3-4ae7-8015-d0a38b9f0eb4 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:53.137033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 680/1332] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:53.142244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc73f86-e2bf-4a79-bee7-aa06af23c3e3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:53.142658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc73f86-e2bf-4a79-bee7-aa06af23c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:53.142805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc73f86-e2bf-4a79-bee7-aa06af23c3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:53.209416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3bc73f86-e2bf-4a79-bee7-aa06af23c3e3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:53.211168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3bc73f86-e2bf-4a79-bee7-aa06af23c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:53.308088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 653/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:53.345091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde403e-e3c3-4ae7-8015-d0a38b9f0eb4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:53.346710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8fde403e-e3c3-4ae7-8015-d0a38b9f0eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:53.418803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:53.454132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde403e-e3c3-4ae7-8015-d0a38b9f0eb4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:53.525589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e346c0d-2d78-486b-a811-6cd1bd14115b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:53.526222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e346c0d-2d78-486b-a811-6cd1bd14115b service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:53.526500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e346c0d-2d78-486b-a811-6cd1bd14115b service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:53.526771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e346c0d-2d78-486b-a811-6cd1bd14115b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:53.528409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9e346c0d-2d78-486b-a811-6cd1bd14115b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:53.635698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e346c0d-2d78-486b-a811-6cd1bd14115b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:53.637073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e346c0d-2d78-486b-a811-6cd1bd14115b service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:53.646736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:53.655364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 681/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:55.752224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255696ca-a5b5-4c95-8df4-c84cfcb445bf None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:55.752815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255696ca-a5b5-4c95-8df4-c84cfcb445bf None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:55.752815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255696ca-a5b5-4c95-8df4-c84cfcb445bf None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:55.754185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 654/1335] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:55.759434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231bdb8-4620-48af-bcbd-5cd1072e876a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:55.759810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231bdb8-4620-48af-bcbd-5cd1072e876a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:55.760023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231bdb8-4620-48af-bcbd-5cd1072e876a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:55.824277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5231bdb8-4620-48af-bcbd-5cd1072e876a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:55.825497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5231bdb8-4620-48af-bcbd-5cd1072e876a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:55.923046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 682/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:55.962688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255696ca-a5b5-4c95-8df4-c84cfcb445bf None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:55.964205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-255696ca-a5b5-4c95-8df4-c84cfcb445bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:56.003306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:56.067328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255696ca-a5b5-4c95-8df4-c84cfcb445bf None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:56.138558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec25af-d419-4bef-b016-84707a2f3299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:56.139196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec25af-d419-4bef-b016-84707a2f3299 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:56.139475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec25af-d419-4bef-b016-84707a2f3299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:56.139745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec25af-d419-4bef-b016-84707a2f3299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:56.141406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a8ec25af-d419-4bef-b016-84707a2f3299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:56.223314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:56.248857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec25af-d419-4bef-b016-84707a2f3299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:56.250287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec25af-d419-4bef-b016-84707a2f3299 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:56.268620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 655/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:10:58.773145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f413-290b-42ec-991e-c4db3e1081a7 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:58.773791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f413-290b-42ec-991e-c4db3e1081a7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:58.773791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f413-290b-42ec-991e-c4db3e1081a7 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:58.775052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 683/1338] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:10:58.780101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932789b9-27a9-44a2-b636-780146a5ea17 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:58.780536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932789b9-27a9-44a2-b636-780146a5ea17 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:58.780736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932789b9-27a9-44a2-b636-780146a5ea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:58.852761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-932789b9-27a9-44a2-b636-780146a5ea17 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:10:58.853852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-932789b9-27a9-44a2-b636-780146a5ea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:58.967828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 656/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:10:59.004011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9f413-290b-42ec-991e-c4db3e1081a7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:10:59.005616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2cb9f413-290b-42ec-991e-c4db3e1081a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:59.087602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:10:59.116825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9f413-290b-42ec-991e-c4db3e1081a7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:10:59.192138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a6d6-878b-4dbe-b84d-d0eae73fb512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:10:59.192816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a6d6-878b-4dbe-b84d-d0eae73fb512 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:10:59.193099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a6d6-878b-4dbe-b84d-d0eae73fb512 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:10:59.193375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a6d6-878b-4dbe-b84d-d0eae73fb512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:10:59.195573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4d83a6d6-878b-4dbe-b84d-d0eae73fb512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:10:59.311899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a6d6-878b-4dbe-b84d-d0eae73fb512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:10:59.313290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a6d6-878b-4dbe-b84d-d0eae73fb512 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:10:59.332056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 684/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:10:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:11:03.589701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a956a-3201-44c4-a552-3c8cce29cbac None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:03.590236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a956a-3201-44c4-a552-3c8cce29cbac None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:03.590236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a956a-3201-44c4-a552-3c8cce29cbac None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:03.591765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 657/1341] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:03.596791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101760cb-4d52-489a-b328-d8aa680632bb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:03.597239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101760cb-4d52-489a-b328-d8aa680632bb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:03.597336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101760cb-4d52-489a-b328-d8aa680632bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:03.603910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:03.669379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-101760cb-4d52-489a-b328-d8aa680632bb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:03.670614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-101760cb-4d52-489a-b328-d8aa680632bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:03.795613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 685/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:03.840904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a956a-3201-44c4-a552-3c8cce29cbac None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:03.843263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-aa4a956a-3201-44c4-a552-3c8cce29cbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:03.906795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:03.955035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a956a-3201-44c4-a552-3c8cce29cbac None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:03.958647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7ac69-6602-444f-b994-ab4505e800b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:03.959292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7ac69-6602-444f-b994-ab4505e800b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:03.960321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7ac69-6602-444f-b994-ab4505e800b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:03.960869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7ac69-6602-444f-b994-ab4505e800b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:03.970741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7ac69-6602-444f-b994-ab4505e800b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:03.972173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7ac69-6602-444f-b994-ab4505e800b8 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:03.982090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 658/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:06.655868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54aeea-6ced-48ee-9428-2b0578a6cce6 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:06.656412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54aeea-6ced-48ee-9428-2b0578a6cce6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:06.656412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54aeea-6ced-48ee-9428-2b0578a6cce6 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:06.657839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 686/1344] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:06.664579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bba93-825e-434f-a166-d9fe1796395b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:06.665101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bba93-825e-434f-a166-d9fe1796395b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:06.665196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bba93-825e-434f-a166-d9fe1796395b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:06.695227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1d4bba93-825e-434f-a166-d9fe1796395b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:06.696434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1d4bba93-825e-434f-a166-d9fe1796395b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:06.791690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 659/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:06.832629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54aeea-6ced-48ee-9428-2b0578a6cce6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:06.835615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6b54aeea-6ced-48ee-9428-2b0578a6cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:06.854173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54aeea-6ced-48ee-9428-2b0578a6cce6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:06.858581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac4501-b6ca-4b02-bd43-9e6f167fdcff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:06.859253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac4501-b6ca-4b02-bd43-9e6f167fdcff service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:06.859539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac4501-b6ca-4b02-bd43-9e6f167fdcff service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:06.859821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac4501-b6ca-4b02-bd43-9e6f167fdcff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:06.871610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac4501-b6ca-4b02-bd43-9e6f167fdcff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:06.873169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac4501-b6ca-4b02-bd43-9e6f167fdcff service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:06.887082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 687/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:11.258030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e148-59b2-4e85-9678-63d29e15d7c2 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:11.258628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e148-59b2-4e85-9678-63d29e15d7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:11.258628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e148-59b2-4e85-9678-63d29e15d7c2 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:11.259931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 660/1347] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:11.265466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384894-de9c-49b6-b0e3-87428d70db8c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:11.265940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384894-de9c-49b6-b0e3-87428d70db8c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:11.266046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384894-de9c-49b6-b0e3-87428d70db8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:11.285250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-15384894-de9c-49b6-b0e3-87428d70db8c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:11.286504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-15384894-de9c-49b6-b0e3-87428d70db8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:11.294929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 688/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:11.333905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e148-59b2-4e85-9678-63d29e15d7c2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:11.345052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e148-59b2-4e85-9678-63d29e15d7c2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:11.348573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726feb4-a4a3-401b-a325-78387b56b803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:11.349223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726feb4-a4a3-401b-a325-78387b56b803 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:11.349513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726feb4-a4a3-401b-a325-78387b56b803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:11.349822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726feb4-a4a3-401b-a325-78387b56b803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:11.359168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726feb4-a4a3-401b-a325-78387b56b803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:11.360640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726feb4-a4a3-401b-a325-78387b56b803 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:11.375209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 661/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:16.446947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d20db-37a1-4b9b-8ce5-a6765e77bafc None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:16.447449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d20db-37a1-4b9b-8ce5-a6765e77bafc None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:16.447543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d20db-37a1-4b9b-8ce5-a6765e77bafc None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:16.449182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 689/1350] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:16.454293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2d3ba-e0ac-4561-a064-f7dff7df228c None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:16.455597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2d3ba-e0ac-4561-a064-f7dff7df228c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:16.456633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2d3ba-e0ac-4561-a064-f7dff7df228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:16.535911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-66c2d3ba-e0ac-4561-a064-f7dff7df228c None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:16.537889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-66c2d3ba-e0ac-4561-a064-f7dff7df228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:16.607393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:16.674236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 662/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:20.979671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8534-1b16-46ea-9f86-f2184beace16 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:20.980230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8534-1b16-46ea-9f86-f2184beace16 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:20.980230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8534-1b16-46ea-9f86-f2184beace16 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:20.981837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 690/1352] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:20.987003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a209f-0731-417d-891b-41792f0f5944 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:20.987753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a209f-0731-417d-891b-41792f0f5944 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:20.988202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a209f-0731-417d-891b-41792f0f5944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:21.007450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8a6a209f-0731-417d-891b-41792f0f5944 None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:21.008664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8a6a209f-0731-417d-891b-41792f0f5944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:21.017422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 663/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:11:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:21.279162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=72 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:22.599294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771ed74-8783-40a8-bc3a-f5eabf20a851 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:22.600609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771ed74-8783-40a8-bc3a-f5eabf20a851 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:22.600609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771ed74-8783-40a8-bc3a-f5eabf20a851 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:22.602044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 691/1354] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:22.607074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9648c-f8a9-46b2-9857-ceb82f5140c2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:22.607517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9648c-f8a9-46b2-9857-ceb82f5140c2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:22.607767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9648c-f8a9-46b2-9857-ceb82f5140c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:22.625838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-bff9648c-f8a9-46b2-9857-ceb82f5140c2 None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:22.627138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bff9648c-f8a9-46b2-9857-ceb82f5140c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:22.635622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 664/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:11:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:11:26.361536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1353d-fe2d-4594-884d-25c7e3dcc980 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:26.362776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1353d-fe2d-4594-884d-25c7e3dcc980 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:26.363064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1353d-fe2d-4594-884d-25c7e3dcc980 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:26.364795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 692/1356] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:26.373885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4700578-da4e-4621-a60a-e7e2d04ec8b8 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:26.374739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4700578-da4e-4621-a60a-e7e2d04ec8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:26.375204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4700578-da4e-4621-a60a-e7e2d04ec8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:26.399828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-b4700578-da4e-4621-a60a-e7e2d04ec8b8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:26.401312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b4700578-da4e-4621-a60a-e7e2d04ec8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:26.416718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 665/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:26.473918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0fd6b-dc62-4c91-9900-b2de43da48fa None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:26.474222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0fd6b-dc62-4c91-9900-b2de43da48fa None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:26.475087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0fd6b-dc62-4c91-9900-b2de43da48fa None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:26.476434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 693/1358] 10.222.0.52 () {54 vars in 961 bytes} [Fri Jun 24 09:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:26.483062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77477de7-9cc4-4db0-8b0a-e27935afdb75 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:26.483525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77477de7-9cc4-4db0-8b0a-e27935afdb75 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:26.483980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77477de7-9cc4-4db0-8b0a-e27935afdb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:26.549168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-77477de7-9cc4-4db0-8b0a-e27935afdb75 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:26.550622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-77477de7-9cc4-4db0-8b0a-e27935afdb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:26.645738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 666/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Fri Jun 24 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:26.654623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0fd6b-dc62-4c91-9900-b2de43da48fa None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:26.669345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0fd6b-dc62-4c91-9900-b2de43da48fa None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:26.675187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb415-9ada-4b75-b3d1-155e198123f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:26.676074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb415-9ada-4b75-b3d1-155e198123f5 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:26.676511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb415-9ada-4b75-b3d1-155e198123f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:26.676964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb415-9ada-4b75-b3d1-155e198123f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:26.687185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb415-9ada-4b75-b3d1-155e198123f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:26.689293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb415-9ada-4b75-b3d1-155e198123f5 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:26.703416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 694/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:11:28.980659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f860750-9104-4222-b7c2-7905b232f6b6 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:28.981173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f860750-9104-4222-b7c2-7905b232f6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:28.981257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f860750-9104-4222-b7c2-7905b232f6b6 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:28.982754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 667/1361] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:28.987865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3be946-a40e-4b87-89be-e7fd873b8402 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:28.988278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3be946-a40e-4b87-89be-e7fd873b8402 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:28.988419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3be946-a40e-4b87-89be-e7fd873b8402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:29.009312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-aa3be946-a40e-4b87-89be-e7fd873b8402 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:29.010731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-aa3be946-a40e-4b87-89be-e7fd873b8402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:29.028633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 695/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:29.063965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f860750-9104-4222-b7c2-7905b232f6b6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:29.080261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f860750-9104-4222-b7c2-7905b232f6b6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:29.086105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9b78d-fe7e-43b4-92c6-5826e51e55f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:29.086860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9b78d-fe7e-43b4-92c6-5826e51e55f5 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:29.087533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9b78d-fe7e-43b4-92c6-5826e51e55f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:29.087916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9b78d-fe7e-43b4-92c6-5826e51e55f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:29.100804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9b78d-fe7e-43b4-92c6-5826e51e55f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:29.102549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9b78d-fe7e-43b4-92c6-5826e51e55f5 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:29.113370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 668/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:31.487913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98f8bd-b5aa-4b48-b70a-625f8c890489 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:31.488348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98f8bd-b5aa-4b48-b70a-625f8c890489 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:31.488458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98f8bd-b5aa-4b48-b70a-625f8c890489 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:31.489846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 696/1364] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:31.495651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594df94c-9b01-4d73-a458-073ca47503fc None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:31.496058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594df94c-9b01-4d73-a458-073ca47503fc None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:31.496215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594df94c-9b01-4d73-a458-073ca47503fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:31.515472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-594df94c-9b01-4d73-a458-073ca47503fc None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:31.516686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-594df94c-9b01-4d73-a458-073ca47503fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:31.524604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 669/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:31.570775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cfc39-a062-4285-8f13-0884435f25ff None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:31.571268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cfc39-a062-4285-8f13-0884435f25ff None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:31.571707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cfc39-a062-4285-8f13-0884435f25ff None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:31.573569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 697/1366] 10.222.0.52 () {54 vars in 961 bytes} [Fri Jun 24 09:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:31.580837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c2cf-ae7e-45f7-8e52-1ba58887fec7 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:31.581306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c2cf-ae7e-45f7-8e52-1ba58887fec7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:31.581592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c2cf-ae7e-45f7-8e52-1ba58887fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:31.600577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1c57c2cf-ae7e-45f7-8e52-1ba58887fec7 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:31.602198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1c57c2cf-ae7e-45f7-8e52-1ba58887fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:31.620490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 670/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Fri Jun 24 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:31.621491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cfc39-a062-4285-8f13-0884435f25ff None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:31.647943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cfc39-a062-4285-8f13-0884435f25ff None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:31.653941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff42026-18ee-4c74-9f81-b6fd4aabe6f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:31.654594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff42026-18ee-4c74-9f81-b6fd4aabe6f9 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:31.654877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff42026-18ee-4c74-9f81-b6fd4aabe6f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:31.655151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff42026-18ee-4c74-9f81-b6fd4aabe6f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:31.664735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff42026-18ee-4c74-9f81-b6fd4aabe6f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:31.666364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff42026-18ee-4c74-9f81-b6fd4aabe6f9 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:31.676108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 698/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:33.830097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e064a-a0cf-4979-98e2-c41de1e99ccc None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:33.830667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e064a-a0cf-4979-98e2-c41de1e99ccc None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:33.831371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e064a-a0cf-4979-98e2-c41de1e99ccc None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:33.833314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 671/1369] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:33.840098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba5b2b-aa92-4586-8c99-5bde11303331 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:33.840627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba5b2b-aa92-4586-8c99-5bde11303331 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:33.840758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba5b2b-aa92-4586-8c99-5bde11303331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:33.860017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c4ba5b2b-aa92-4586-8c99-5bde11303331 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:33.861288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c4ba5b2b-aa92-4586-8c99-5bde11303331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:33.870119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 699/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:33.916938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e064a-a0cf-4979-98e2-c41de1e99ccc None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:33.932576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e064a-a0cf-4979-98e2-c41de1e99ccc None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:33.938450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a2065-2cf9-4247-85ca-7be265a2a9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:33.939093 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2065-2cf9-4247-85ca-7be265a2a9d8 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:33.939381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2065-2cf9-4247-85ca-7be265a2a9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:33.939658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2065-2cf9-4247-85ca-7be265a2a9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:33.948777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a2065-2cf9-4247-85ca-7be265a2a9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:33.950365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a2065-2cf9-4247-85ca-7be265a2a9d8 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:33.962703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 672/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:34.698384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc9aa-5910-4d71-9ca7-2fe1cb69e884 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:34.698976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc9aa-5910-4d71-9ca7-2fe1cb69e884 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:34.699375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc9aa-5910-4d71-9ca7-2fe1cb69e884 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:34.700914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 700/1372] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:34.706302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d22b35-6ed7-45ce-b168-34df4ed384d1 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:34.706802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d22b35-6ed7-45ce-b168-34df4ed384d1 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:34.707238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d22b35-6ed7-45ce-b168-34df4ed384d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:34.726560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-63d22b35-6ed7-45ce-b168-34df4ed384d1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:34.727747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-63d22b35-6ed7-45ce-b168-34df4ed384d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:34.735932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 673/1373] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:35.059126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac0f3-d8a6-4995-88c4-1a74d4e0b38d None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:35.060034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac0f3-d8a6-4995-88c4-1a74d4e0b38d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:35.060764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac0f3-d8a6-4995-88c4-1a74d4e0b38d None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:35.062456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 701/1374] 10.222.0.52 () {54 vars in 963 bytes} [Fri Jun 24 09:11:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:35.069153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b1ed-f5fd-4682-85d6-9bad0706ba7b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:35.069752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b1ed-f5fd-4682-85d6-9bad0706ba7b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:35.070012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b1ed-f5fd-4682-85d6-9bad0706ba7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:35.076641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:35.122748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f946b1ed-f5fd-4682-85d6-9bad0706ba7b None None] MFA Rules not processed for user `31bf0e50e20b47d0a63662bbecce833d`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:35.123978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f946b1ed-f5fd-4682-85d6-9bad0706ba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:35.190725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 674/1375] 10.222.0.52 () {58 vars in 1060 bytes} [Fri Jun 24 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:35.199034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfac0f3-d8a6-4995-88c4-1a74d4e0b38d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:35.211058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfac0f3-d8a6-4995-88c4-1a74d4e0b38d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:35.214827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb4137-bc4c-465b-93d9-56346e0963e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:35.215470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb4137-bc4c-465b-93d9-56346e0963e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:35.215757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb4137-bc4c-465b-93d9-56346e0963e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:35.216038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb4137-bc4c-465b-93d9-56346e0963e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:35.225595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb4137-bc4c-465b-93d9-56346e0963e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:35.230113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb4137-bc4c-465b-93d9-56346e0963e5 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:35.244052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 702/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:36.089526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43166d4-34fe-4896-a5dc-502b7ac2b6c4 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:36.090025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43166d4-34fe-4896-a5dc-502b7ac2b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:36.090148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43166d4-34fe-4896-a5dc-502b7ac2b6c4 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:36.091460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 675/1377] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:36.097221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b00e-c048-4bc4-90d1-cab93d9e21e3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:36.097670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b00e-c048-4bc4-90d1-cab93d9e21e3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:36.097988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b00e-c048-4bc4-90d1-cab93d9e21e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:36.127841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-db06b00e-c048-4bc4-90d1-cab93d9e21e3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:36.129094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-db06b00e-c048-4bc4-90d1-cab93d9e21e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:36.137588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 703/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:36.181374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43166d4-34fe-4896-a5dc-502b7ac2b6c4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:36.196651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43166d4-34fe-4896-a5dc-502b7ac2b6c4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:36.201972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c61a-0677-4f56-9df7-c9e31cdae133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:36.202601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c61a-0677-4f56-9df7-c9e31cdae133 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:36.202878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c61a-0677-4f56-9df7-c9e31cdae133 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:36.203157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c61a-0677-4f56-9df7-c9e31cdae133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:36.211215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c61a-0677-4f56-9df7-c9e31cdae133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:36.212745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c61a-0677-4f56-9df7-c9e31cdae133 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:36.222015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 676/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:38.351066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71943-d621-4f0f-bbe7-7c572751c934 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:38.351608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71943-d621-4f0f-bbe7-7c572751c934 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:38.351608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf71943-d621-4f0f-bbe7-7c572751c934 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:38.353057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 704/1380] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:38.358283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9460c-0c57-4783-a76e-6e6eb35716d5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:38.358726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9460c-0c57-4783-a76e-6e6eb35716d5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:38.358850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9460c-0c57-4783-a76e-6e6eb35716d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:38.377436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-83c9460c-0c57-4783-a76e-6e6eb35716d5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:38.378628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-83c9460c-0c57-4783-a76e-6e6eb35716d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:38.387708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 677/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:38.434849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf71943-d621-4f0f-bbe7-7c572751c934 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:38.450708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf71943-d621-4f0f-bbe7-7c572751c934 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:38.456720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4063b-7225-49c5-9340-02bc85a32239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:38.457504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4063b-7225-49c5-9340-02bc85a32239 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:38.457792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4063b-7225-49c5-9340-02bc85a32239 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:38.458069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4063b-7225-49c5-9340-02bc85a32239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:38.468880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4063b-7225-49c5-9340-02bc85a32239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:38.470467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4063b-7225-49c5-9340-02bc85a32239 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:38.480149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 705/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:40.593013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728c9fa-7389-4889-bcf3-a8bce736bfad None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:40.593921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728c9fa-7389-4889-bcf3-a8bce736bfad None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:40.594340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728c9fa-7389-4889-bcf3-a8bce736bfad None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:40.596047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 678/1383] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:40.602157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5b97b-aa0d-4296-b455-c038c6adb0b0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:40.602648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5b97b-aa0d-4296-b455-c038c6adb0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:40.602784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5b97b-aa0d-4296-b455-c038c6adb0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:40.621201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6bd5b97b-aa0d-4296-b455-c038c6adb0b0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:40.622398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6bd5b97b-aa0d-4296-b455-c038c6adb0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:40.632874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 706/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:11:40.672755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728c9fa-7389-4889-bcf3-a8bce736bfad None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:40.688961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728c9fa-7389-4889-bcf3-a8bce736bfad None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:40.694944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7061e2-1b9e-42a4-8910-07f246be6bfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:40.695638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7061e2-1b9e-42a4-8910-07f246be6bfd service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:40.695931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7061e2-1b9e-42a4-8910-07f246be6bfd service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:40.696207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7061e2-1b9e-42a4-8910-07f246be6bfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:40.705470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7061e2-1b9e-42a4-8910-07f246be6bfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:40.707051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7061e2-1b9e-42a4-8910-07f246be6bfd service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:40.719099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 679/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:42.821090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e482aca-d3bc-4e2d-b75c-6567e94ba679 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:42.821090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e482aca-d3bc-4e2d-b75c-6567e94ba679 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:42.821972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e482aca-d3bc-4e2d-b75c-6567e94ba679 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:42.828097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5d6d-b94a-406e-a3e4-64a18cb0e597 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:42.828707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5d6d-b94a-406e-a3e4-64a18cb0e597 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:42.828707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5d6d-b94a-406e-a3e4-64a18cb0e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:42.830589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 707/1386] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:42.848168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-424f5d6d-b94a-406e-a3e4-64a18cb0e597 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:42.849604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-424f5d6d-b94a-406e-a3e4-64a18cb0e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:42.858699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 680/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:11:42.902992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e482aca-d3bc-4e2d-b75c-6567e94ba679 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:42.923252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e482aca-d3bc-4e2d-b75c-6567e94ba679 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:42.930910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b772d-d86c-4dfb-86ee-766eb8cd6976 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:42.931569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b772d-d86c-4dfb-86ee-766eb8cd6976 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:42.931878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b772d-d86c-4dfb-86ee-766eb8cd6976 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:42.932152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b772d-d86c-4dfb-86ee-766eb8cd6976 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:42.941686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b772d-d86c-4dfb-86ee-766eb8cd6976 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:42.943698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b772d-d86c-4dfb-86ee-766eb8cd6976 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:42.953589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 708/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:45.355654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55ba8-a0c6-4c93-a3a2-ffe8c12a093a None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:45.356294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55ba8-a0c6-4c93-a3a2-ffe8c12a093a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:45.356813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55ba8-a0c6-4c93-a3a2-ffe8c12a093a None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:45.359159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 681/1389] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:45.364412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034bfe-a587-441d-8214-de9f9a7622e5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:45.364873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034bfe-a587-441d-8214-de9f9a7622e5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:45.365043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034bfe-a587-441d-8214-de9f9a7622e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:45.383372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cd034bfe-a587-441d-8214-de9f9a7622e5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:45.384656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cd034bfe-a587-441d-8214-de9f9a7622e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:45.392958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 709/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:45.441104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a55ba8-a0c6-4c93-a3a2-ffe8c12a093a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:45.455744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a55ba8-a0c6-4c93-a3a2-ffe8c12a093a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:45.461857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad430f4-e681-4c56-af9b-a9fbb4ec756a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:45.462507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad430f4-e681-4c56-af9b-a9fbb4ec756a service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:45.462790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad430f4-e681-4c56-af9b-a9fbb4ec756a service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:45.463075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad430f4-e681-4c56-af9b-a9fbb4ec756a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:45.471567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad430f4-e681-4c56-af9b-a9fbb4ec756a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:45.473200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad430f4-e681-4c56-af9b-a9fbb4ec756a service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:45.483218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 682/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:11:47.668147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951980e3-f49a-4711-9bbc-c0a536f15806 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:47.668778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951980e3-f49a-4711-9bbc-c0a536f15806 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:47.668778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951980e3-f49a-4711-9bbc-c0a536f15806 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:47.670068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 710/1392] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:11:47.675647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fa26c-e8b9-40c3-840b-8a253219a29d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:47.676070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fa26c-e8b9-40c3-840b-8a253219a29d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:47.676207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fa26c-e8b9-40c3-840b-8a253219a29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:47.695645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-cb9fa26c-e8b9-40c3-840b-8a253219a29d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:47.696930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cb9fa26c-e8b9-40c3-840b-8a253219a29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:47.704807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 683/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:47.753421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951980e3-f49a-4711-9bbc-c0a536f15806 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:47.768012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951980e3-f49a-4711-9bbc-c0a536f15806 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:47.773823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91459f64-ba70-4f10-a9ed-1c0561d83bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:47.774459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459f64-ba70-4f10-a9ed-1c0561d83bed service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:47.774757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459f64-ba70-4f10-a9ed-1c0561d83bed service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:47.775010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459f64-ba70-4f10-a9ed-1c0561d83bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:47.785342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459f64-ba70-4f10-a9ed-1c0561d83bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:47.787953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459f64-ba70-4f10-a9ed-1c0561d83bed service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:47.798604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 711/1394] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:50.862438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff484-2360-476e-9183-b9b2713322ff None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:50.862982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff484-2360-476e-9183-b9b2713322ff None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:50.862982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff484-2360-476e-9183-b9b2713322ff None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:50.865003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 684/1395] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:50.870394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3643e9-9f18-42ad-872b-5a042b5649c0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:50.870791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3643e9-9f18-42ad-872b-5a042b5649c0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:50.870956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3643e9-9f18-42ad-872b-5a042b5649c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:50.888995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0d3643e9-9f18-42ad-872b-5a042b5649c0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:50.890226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0d3643e9-9f18-42ad-872b-5a042b5649c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:50.898124 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 712/1396] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:50.943182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ff484-2360-476e-9183-b9b2713322ff None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:50.958299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ff484-2360-476e-9183-b9b2713322ff None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:50.963690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc2533-b8b5-4ae9-b8bd-c31f5817e98f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:50.964330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc2533-b8b5-4ae9-b8bd-c31f5817e98f service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:50.964641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc2533-b8b5-4ae9-b8bd-c31f5817e98f service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:50.964920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc2533-b8b5-4ae9-b8bd-c31f5817e98f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:50.973333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc2533-b8b5-4ae9-b8bd-c31f5817e98f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:50.975463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc2533-b8b5-4ae9-b8bd-c31f5817e98f service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:50.984925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 685/1397] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:11:51.110761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615dfcc-d828-463b-85b6-4cdeb6165e75 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:51.111055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615dfcc-d828-463b-85b6-4cdeb6165e75 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:51.111333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615dfcc-d828-463b-85b6-4cdeb6165e75 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:51.112861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 713/1398] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:51.118605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de368825-64c7-4427-8e45-37ea38dd4802 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:51.118983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de368825-64c7-4427-8e45-37ea38dd4802 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:51.119324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de368825-64c7-4427-8e45-37ea38dd4802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:51.138856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-de368825-64c7-4427-8e45-37ea38dd4802 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:51.140076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-de368825-64c7-4427-8e45-37ea38dd4802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:51.148791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 686/1399] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:51.209330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e330-ec63-4f01-a1f3-065bed038776 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:51.209618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e330-ec63-4f01-a1f3-065bed038776 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:51.209895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e330-ec63-4f01-a1f3-065bed038776 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:51.211236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 714/1400] 10.222.0.52 () {54 vars in 962 bytes} [Fri Jun 24 09:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:51.216374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746d121-be8f-4548-9fb9-588e4fd42cbf None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:51.216769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746d121-be8f-4548-9fb9-588e4fd42cbf None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:51.217052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746d121-be8f-4548-9fb9-588e4fd42cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:51.266542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9746d121-be8f-4548-9fb9-588e4fd42cbf None None] MFA Rules not processed for user `28a07f47b8954418a5a86f344fb7fbe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:51.267728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9746d121-be8f-4548-9fb9-588e4fd42cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:51.301227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:51.333994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 687/1401] 10.222.0.52 () {58 vars in 1059 bytes} [Fri Jun 24 09:11:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:51.341611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e330-ec63-4f01-a1f3-065bed038776 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:51.346932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=55 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:51.353708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e330-ec63-4f01-a1f3-065bed038776 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:51.357584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b108cb2-4d56-4b6d-b257-8dc865f0efd3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:51.358219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108cb2-4d56-4b6d-b257-8dc865f0efd3 service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:51.358526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108cb2-4d56-4b6d-b257-8dc865f0efd3 service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:51.358798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108cb2-4d56-4b6d-b257-8dc865f0efd3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:51.368634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b108cb2-4d56-4b6d-b257-8dc865f0efd3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:51.370011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b108cb2-4d56-4b6d-b257-8dc865f0efd3 service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:51.379429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 715/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:11:52.115894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8337ab-4db0-4bb3-bfb8-63157c816521 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:52.116456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8337ab-4db0-4bb3-bfb8-63157c816521 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:52.116578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8337ab-4db0-4bb3-bfb8-63157c816521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:52.173546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ed8337ab-4db0-4bb3-bfb8-63157c816521 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:52.174727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ed8337ab-4db0-4bb3-bfb8-63157c816521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:52.261227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 688/1403] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:52.308281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b657-de80-471a-a8ab-31cef840d6d1 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:52.308821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b657-de80-471a-a8ab-31cef840d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:52.308957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b657-de80-471a-a8ab-31cef840d6d1 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:52.310700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 716/1404] 10.222.0.52 () {54 vars in 942 bytes} [Fri Jun 24 09:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:52.317880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ef586-75b1-49fa-97f8-c8478774cfac None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:52.318171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ef586-75b1-49fa-97f8-c8478774cfac None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:52.318453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ef586-75b1-49fa-97f8-c8478774cfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:52.366066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-aa0ef586-75b1-49fa-97f8-c8478774cfac None None] MFA Rules not processed for user `cbaf3f85739348beb1bc8bae579f0da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:52.367419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-aa0ef586-75b1-49fa-97f8-c8478774cfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:52.437788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 689/1405] 10.222.0.52 () {58 vars in 1039 bytes} [Fri Jun 24 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:52.447886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b657-de80-471a-a8ab-31cef840d6d1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:52.461965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b657-de80-471a-a8ab-31cef840d6d1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:52.466939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2fd56-aa02-4f33-b3e1-be3c11a83e88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:52.467661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2fd56-aa02-4f33-b3e1-be3c11a83e88 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:52.468013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2fd56-aa02-4f33-b3e1-be3c11a83e88 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:52.468361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2fd56-aa02-4f33-b3e1-be3c11a83e88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:52.476748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2fd56-aa02-4f33-b3e1-be3c11a83e88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:52.478127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2fd56-aa02-4f33-b3e1-be3c11a83e88 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:52.486234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 717/1406] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:11:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:11:53.110963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bf344-8624-4570-b802-2b2d43d4d1f5 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:53.111480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bf344-8624-4570-b802-2b2d43d4d1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:53.111998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bf344-8624-4570-b802-2b2d43d4d1f5 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:53.114888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 690/1407] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:11:53.120519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c5c7f-7f51-4cd9-9004-63e4ad877afd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:53.120839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c5c7f-7f51-4cd9-9004-63e4ad877afd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:53.121128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c5c7f-7f51-4cd9-9004-63e4ad877afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:53.140474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a04c5c7f-7f51-4cd9-9004-63e4ad877afd None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:11:53.141716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a04c5c7f-7f51-4cd9-9004-63e4ad877afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:11:53.149910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 718/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:11:53.196275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bf344-8624-4570-b802-2b2d43d4d1f5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:11:53.212753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bf344-8624-4570-b802-2b2d43d4d1f5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:11:53.218355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f2d54-0a9e-43a6-a683-4edcc24f7e36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:11:53.219131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f2d54-0a9e-43a6-a683-4edcc24f7e36 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:11:53.219492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f2d54-0a9e-43a6-a683-4edcc24f7e36 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:11:53.219836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f2d54-0a9e-43a6-a683-4edcc24f7e36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:11:53.233046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f2d54-0a9e-43a6-a683-4edcc24f7e36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:11:53.234667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f2d54-0a9e-43a6-a683-4edcc24f7e36 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:11:53.241971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:11:53.248024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 691/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:11:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:03.134479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=14 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:12:14.868240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed4a09-f3b2-40e7-aa4a-61fc61292d77 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:14.868914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed4a09-f3b2-40e7-aa4a-61fc61292d77 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:14.868914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed4a09-f3b2-40e7-aa4a-61fc61292d77 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:14.870206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 719/1410] 10.222.0.52 () {54 vars in 918 bytes} [Fri Jun 24 09:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:14.875557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d619-da9c-42ed-b770-d0beb529f192 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:14.876076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d619-da9c-42ed-b770-d0beb529f192 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:14.876194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d619-da9c-42ed-b770-d0beb529f192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:14.895750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9110d619-da9c-42ed-b770-d0beb529f192 None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:14.897005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9110d619-da9c-42ed-b770-d0beb529f192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:14.905678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 692/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Fri Jun 24 09:12:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:14.920935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768917bd-ef47-4025-8bf0-ffbfec0d726e None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:14.921230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768917bd-ef47-4025-8bf0-ffbfec0d726e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:14.921510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768917bd-ef47-4025-8bf0-ffbfec0d726e None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:14.922877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 720/1412] 10.222.0.52 () {54 vars in 918 bytes} [Fri Jun 24 09:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:14.928289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169dcb4-6043-4476-8f9d-f55156172a45 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:14.928604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169dcb4-6043-4476-8f9d-f55156172a45 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:14.928889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169dcb4-6043-4476-8f9d-f55156172a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:14.947406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a169dcb4-6043-4476-8f9d-f55156172a45 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:14.948654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a169dcb4-6043-4476-8f9d-f55156172a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:14.958449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 693/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Fri Jun 24 09:12:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:17.526354 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98737-5b5b-433d-8d42-2bf18817891c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:17.526907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98737-5b5b-433d-8d42-2bf18817891c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:17.526907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98737-5b5b-433d-8d42-2bf18817891c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:17.528277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 721/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:17.533972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be45a-f272-4930-9ff1-f0106d499220 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:17.535071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be45a-f272-4930-9ff1-f0106d499220 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:17.535181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be45a-f272-4930-9ff1-f0106d499220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:17.554996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-222be45a-f272-4930-9ff1-f0106d499220 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:17.556154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-222be45a-f272-4930-9ff1-f0106d499220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:17.564309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 694/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:17.608413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a98737-5b5b-433d-8d42-2bf18817891c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:17.625314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a98737-5b5b-433d-8d42-2bf18817891c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:17.631313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93198c4d-ddcc-4d42-86fa-14e1c3d05a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:17.631964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93198c4d-ddcc-4d42-86fa-14e1c3d05a2e service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:17.632248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93198c4d-ddcc-4d42-86fa-14e1c3d05a2e service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:17.632523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93198c4d-ddcc-4d42-86fa-14e1c3d05a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:17.641706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93198c4d-ddcc-4d42-86fa-14e1c3d05a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:17.643283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93198c4d-ddcc-4d42-86fa-14e1c3d05a2e service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:17.653598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 722/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:18.572062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c19a-42a4-48dd-b4da-03611ab2c33b None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:18.572795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c19a-42a4-48dd-b4da-03611ab2c33b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:18.572795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c19a-42a4-48dd-b4da-03611ab2c33b None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:18.574177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 695/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:18.579286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27412-1891-4f41-8c5b-ddb7b6d38523 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:18.579704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27412-1891-4f41-8c5b-ddb7b6d38523 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:18.579847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb27412-1891-4f41-8c5b-ddb7b6d38523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:18.598180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7cb27412-1891-4f41-8c5b-ddb7b6d38523 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:18.599387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7cb27412-1891-4f41-8c5b-ddb7b6d38523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:18.607260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 723/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:18.760399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878c19a-42a4-48dd-b4da-03611ab2c33b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:18.774628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878c19a-42a4-48dd-b4da-03611ab2c33b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:18.779921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c03ee-0b40-4c82-9083-f41ab2f856c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:18.780579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c03ee-0b40-4c82-9083-f41ab2f856c7 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:18.780864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c03ee-0b40-4c82-9083-f41ab2f856c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:18.781136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c03ee-0b40-4c82-9083-f41ab2f856c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:18.790320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c03ee-0b40-4c82-9083-f41ab2f856c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:18.791926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c03ee-0b40-4c82-9083-f41ab2f856c7 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:18.802274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 696/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:19.710348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001d427-e835-4cae-a625-8aed2fdd0817 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:19.711000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001d427-e835-4cae-a625-8aed2fdd0817 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:19.711000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001d427-e835-4cae-a625-8aed2fdd0817 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:19.712190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 724/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:19.717728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622851-84a7-4286-83e0-8c606e573bb8 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:19.718195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622851-84a7-4286-83e0-8c606e573bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:19.718400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83622851-84a7-4286-83e0-8c606e573bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:19.737551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-83622851-84a7-4286-83e0-8c606e573bb8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:19.738760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-83622851-84a7-4286-83e0-8c606e573bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:19.746691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 697/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:19.901988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001d427-e835-4cae-a625-8aed2fdd0817 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:19.916838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001d427-e835-4cae-a625-8aed2fdd0817 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:19.923006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be77284-31f2-455d-ac01-f781abf703d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:19.923663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be77284-31f2-455d-ac01-f781abf703d0 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:19.923945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be77284-31f2-455d-ac01-f781abf703d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:19.924225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be77284-31f2-455d-ac01-f781abf703d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:19.933646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be77284-31f2-455d-ac01-f781abf703d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:19.935516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be77284-31f2-455d-ac01-f781abf703d0 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:19.946314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 725/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:20.834169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ec288-4590-4e64-83e7-35f8285309fd None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:20.834718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ec288-4590-4e64-83e7-35f8285309fd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:20.834718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ec288-4590-4e64-83e7-35f8285309fd None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:20.836085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 698/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:20.841236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fe8d4-ddfa-4ddb-a9a6-98f8b3aec186 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:20.841703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fe8d4-ddfa-4ddb-a9a6-98f8b3aec186 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:20.841795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fe8d4-ddfa-4ddb-a9a6-98f8b3aec186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:20.860935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-084fe8d4-ddfa-4ddb-a9a6-98f8b3aec186 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:20.862157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-084fe8d4-ddfa-4ddb-a9a6-98f8b3aec186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:20.870752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 726/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:21.029945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ec288-4590-4e64-83e7-35f8285309fd None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:21.045085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ec288-4590-4e64-83e7-35f8285309fd None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:21.053565 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bdd92-5a50-4345-b830-a70faf680b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:21.054215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdd92-5a50-4345-b830-a70faf680b08 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:21.054497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdd92-5a50-4345-b830-a70faf680b08 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:21.054772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdd92-5a50-4345-b830-a70faf680b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:21.063621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bdd92-5a50-4345-b830-a70faf680b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:21.065251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bdd92-5a50-4345-b830-a70faf680b08 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:21.075236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 699/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:21.951955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb224f5-cacb-4704-870d-a630d10ccdde None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:21.952687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb224f5-cacb-4704-870d-a630d10ccdde None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:21.952687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb224f5-cacb-4704-870d-a630d10ccdde None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:21.953912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 727/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:21.959711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8b068-fea7-4411-8c2f-a287f7da885f None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:21.960171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8b068-fea7-4411-8c2f-a287f7da885f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:21.960280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8b068-fea7-4411-8c2f-a287f7da885f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:21.987975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d7f8b068-fea7-4411-8c2f-a287f7da885f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:21.989207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d7f8b068-fea7-4411-8c2f-a287f7da885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:21.997022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 700/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:22.149979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb224f5-cacb-4704-870d-a630d10ccdde None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:22.165292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb224f5-cacb-4704-870d-a630d10ccdde None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:22.170752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3614e0a-b1f7-43d8-af15-766b67e17e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:22.171399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3614e0a-b1f7-43d8-af15-766b67e17e82 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:22.171679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3614e0a-b1f7-43d8-af15-766b67e17e82 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:22.171953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3614e0a-b1f7-43d8-af15-766b67e17e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:22.180679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3614e0a-b1f7-43d8-af15-766b67e17e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:22.182253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3614e0a-b1f7-43d8-af15-766b67e17e82 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:22.192781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 728/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:12:23.022057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4399-5924-4385-9470-74c731c26871 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:23.022637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4399-5924-4385-9470-74c731c26871 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:23.022637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4399-5924-4385-9470-74c731c26871 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:23.023993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 701/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:23.029255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7bd3b-f0d2-4f47-8b12-f86feaca3bc4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:23.029695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7bd3b-f0d2-4f47-8b12-f86feaca3bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:23.029815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7bd3b-f0d2-4f47-8b12-f86feaca3bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:23.048467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-00a7bd3b-f0d2-4f47-8b12-f86feaca3bc4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:23.049695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-00a7bd3b-f0d2-4f47-8b12-f86feaca3bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:23.058069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 729/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:12:23.212491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec4399-5924-4385-9470-74c731c26871 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:23.228160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec4399-5924-4385-9470-74c731c26871 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:23.233666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de325411-0a76-4998-8f52-b54d8cd8dc56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:23.234587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325411-0a76-4998-8f52-b54d8cd8dc56 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:23.234871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325411-0a76-4998-8f52-b54d8cd8dc56 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:23.235143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325411-0a76-4998-8f52-b54d8cd8dc56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:23.243926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de325411-0a76-4998-8f52-b54d8cd8dc56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:23.245543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de325411-0a76-4998-8f52-b54d8cd8dc56 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:23.255230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 702/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:24.070051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436c33d-344c-4144-834b-7fc7081f3305 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:24.070661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436c33d-344c-4144-834b-7fc7081f3305 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:24.070661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436c33d-344c-4144-834b-7fc7081f3305 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:24.071951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 730/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:24.077001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e0a8-7a6c-4add-92ba-a0868e412123 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:24.077421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e0a8-7a6c-4add-92ba-a0868e412123 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:24.077565 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e0a8-7a6c-4add-92ba-a0868e412123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:24.095118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c738e0a8-7a6c-4add-92ba-a0868e412123 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:24.096315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c738e0a8-7a6c-4add-92ba-a0868e412123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:24.104465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 703/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:24.258432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436c33d-344c-4144-834b-7fc7081f3305 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:24.273274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436c33d-344c-4144-834b-7fc7081f3305 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:24.278664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77194eb1-638e-488d-a951-5d21ccf5bb1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:24.279301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194eb1-638e-488d-a951-5d21ccf5bb1c service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:24.279579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194eb1-638e-488d-a951-5d21ccf5bb1c service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:24.279851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194eb1-638e-488d-a951-5d21ccf5bb1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:24.288544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77194eb1-638e-488d-a951-5d21ccf5bb1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:24.290077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77194eb1-638e-488d-a951-5d21ccf5bb1c service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:24.300366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 731/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:12:25.110637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df27637-be16-48ef-b6e9-7c28404177e9 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:25.111205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df27637-be16-48ef-b6e9-7c28404177e9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:25.111205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df27637-be16-48ef-b6e9-7c28404177e9 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:25.112654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 704/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:25.117984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69005799-8503-4698-bd53-d4126b41999e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:25.118618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69005799-8503-4698-bd53-d4126b41999e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:25.118618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69005799-8503-4698-bd53-d4126b41999e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:25.137274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-69005799-8503-4698-bd53-d4126b41999e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:25.138519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-69005799-8503-4698-bd53-d4126b41999e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:25.147155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 732/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:25.297703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df27637-be16-48ef-b6e9-7c28404177e9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:25.312610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df27637-be16-48ef-b6e9-7c28404177e9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:25.318941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2ac84-5e96-4a80-afaf-6a9f57c22135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:25.319590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2ac84-5e96-4a80-afaf-6a9f57c22135 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:25.319872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2ac84-5e96-4a80-afaf-6a9f57c22135 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:25.320325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2ac84-5e96-4a80-afaf-6a9f57c22135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:25.329240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2ac84-5e96-4a80-afaf-6a9f57c22135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:25.331433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2ac84-5e96-4a80-afaf-6a9f57c22135 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:25.341621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 705/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:12:26.157055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb597b1-09e6-4bc1-85dd-b59b16dd6415 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:26.157650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb597b1-09e6-4bc1-85dd-b59b16dd6415 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:26.157650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb597b1-09e6-4bc1-85dd-b59b16dd6415 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:26.159524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 733/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:12:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:26.164909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ab00c-da41-46d3-a511-e40ce587d51f None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:26.165399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ab00c-da41-46d3-a511-e40ce587d51f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:26.165488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ab00c-da41-46d3-a511-e40ce587d51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:26.184852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-237ab00c-da41-46d3-a511-e40ce587d51f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:26.186099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-237ab00c-da41-46d3-a511-e40ce587d51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:26.193993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 706/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:26.347646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb597b1-09e6-4bc1-85dd-b59b16dd6415 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:26.362139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb597b1-09e6-4bc1-85dd-b59b16dd6415 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:26.368576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32a62a-a87d-475c-a789-66f1df71d647 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:26.369251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a62a-a87d-475c-a789-66f1df71d647 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:26.369534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a62a-a87d-475c-a789-66f1df71d647 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:26.369806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32a62a-a87d-475c-a789-66f1df71d647 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:26.378434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32a62a-a87d-475c-a789-66f1df71d647 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:26.380205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32a62a-a87d-475c-a789-66f1df71d647 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:26.390398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 734/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:28.340391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e65cf9-82bb-49d8-a62f-1902dab9f315 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:28.340952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e65cf9-82bb-49d8-a62f-1902dab9f315 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:28.341069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e65cf9-82bb-49d8-a62f-1902dab9f315 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:28.342386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 707/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:28.347891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838fba-3f20-4442-92fc-83d0329c13ab None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:28.348347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838fba-3f20-4442-92fc-83d0329c13ab None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:28.348482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838fba-3f20-4442-92fc-83d0329c13ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:28.366551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5c838fba-3f20-4442-92fc-83d0329c13ab None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:28.367769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5c838fba-3f20-4442-92fc-83d0329c13ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:28.375792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 735/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:28.415774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e65cf9-82bb-49d8-a62f-1902dab9f315 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:28.426574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e65cf9-82bb-49d8-a62f-1902dab9f315 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:28.430634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5715e-3e42-462c-83ec-703d0921c341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:28.431281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5715e-3e42-462c-83ec-703d0921c341 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:28.431810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5715e-3e42-462c-83ec-703d0921c341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:28.432137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5715e-3e42-462c-83ec-703d0921c341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:28.442762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5715e-3e42-462c-83ec-703d0921c341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:28.444190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5715e-3e42-462c-83ec-703d0921c341 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:28.453824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 708/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:30.691022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148e6b-f3f3-4c7a-9c16-e85348e30edf None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:30.691572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148e6b-f3f3-4c7a-9c16-e85348e30edf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:30.691572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148e6b-f3f3-4c7a-9c16-e85348e30edf None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:30.692919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 736/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:30.698710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6e43f-672c-4da4-9b07-fd2a5571c72e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:30.699202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6e43f-672c-4da4-9b07-fd2a5571c72e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:30.699298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6e43f-672c-4da4-9b07-fd2a5571c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:30.717868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-91f6e43f-672c-4da4-9b07-fd2a5571c72e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:30.719073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-91f6e43f-672c-4da4-9b07-fd2a5571c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:30.727772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 709/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:30.765734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148e6b-f3f3-4c7a-9c16-e85348e30edf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:30.778190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148e6b-f3f3-4c7a-9c16-e85348e30edf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:30.781706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e86bf-4c6e-4738-b8f6-9ce0c1b2cdfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:30.782337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e86bf-4c6e-4738-b8f6-9ce0c1b2cdfb service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:30.782615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e86bf-4c6e-4738-b8f6-9ce0c1b2cdfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:30.782887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e86bf-4c6e-4738-b8f6-9ce0c1b2cdfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:30.792996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e86bf-4c6e-4738-b8f6-9ce0c1b2cdfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:30.794459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e86bf-4c6e-4738-b8f6-9ce0c1b2cdfb service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:30.804461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 737/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:33.278077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f016e29-ba17-4259-b02d-c27f2a3bbb57 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:33.278643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f016e29-ba17-4259-b02d-c27f2a3bbb57 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:33.278643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f016e29-ba17-4259-b02d-c27f2a3bbb57 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:33.279993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 710/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:33.286147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc4c05-29a2-4421-b825-70db7792fb1e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:33.288923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc4c05-29a2-4421-b825-70db7792fb1e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:33.288923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc4c05-29a2-4421-b825-70db7792fb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:33.304956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fbcc4c05-29a2-4421-b825-70db7792fb1e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:33.306167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fbcc4c05-29a2-4421-b825-70db7792fb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:33.314449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 738/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:33.354400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f016e29-ba17-4259-b02d-c27f2a3bbb57 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:33.369562 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f016e29-ba17-4259-b02d-c27f2a3bbb57 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:33.375137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a84d1-ebf9-40ce-a015-ad808a26af4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:33.375802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a84d1-ebf9-40ce-a015-ad808a26af4a service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:33.376091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a84d1-ebf9-40ce-a015-ad808a26af4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:33.376376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a84d1-ebf9-40ce-a015-ad808a26af4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:33.387671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a84d1-ebf9-40ce-a015-ad808a26af4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:33.389163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a84d1-ebf9-40ce-a015-ad808a26af4a service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:33.399490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 711/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:35.596182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359f87a-735b-4ecc-86bd-cb0ef734969c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:35.596747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359f87a-735b-4ecc-86bd-cb0ef734969c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:35.596747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359f87a-735b-4ecc-86bd-cb0ef734969c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:35.598090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 739/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:35.603460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38540a28-6c56-4ab2-b660-28e99a70b804 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:35.603862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38540a28-6c56-4ab2-b660-28e99a70b804 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:35.604026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38540a28-6c56-4ab2-b660-28e99a70b804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:35.622783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-38540a28-6c56-4ab2-b660-28e99a70b804 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:35.624168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-38540a28-6c56-4ab2-b660-28e99a70b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:35.632646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 712/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:35.681984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359f87a-735b-4ecc-86bd-cb0ef734969c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:35.693919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359f87a-735b-4ecc-86bd-cb0ef734969c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:35.697744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2eef5-47a2-40ec-8219-cdbd0eab7a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:35.698459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2eef5-47a2-40ec-8219-cdbd0eab7a5c service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:35.698746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2eef5-47a2-40ec-8219-cdbd0eab7a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:35.699025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2eef5-47a2-40ec-8219-cdbd0eab7a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:35.709739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2eef5-47a2-40ec-8219-cdbd0eab7a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:35.711250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2eef5-47a2-40ec-8219-cdbd0eab7a5c service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:35.723238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 740/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:37.887679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6501afc-9f01-421d-b59f-faadc5e8a3d4 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:37.888497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6501afc-9f01-421d-b59f-faadc5e8a3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:37.888838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6501afc-9f01-421d-b59f-faadc5e8a3d4 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:37.890383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 713/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:37.895970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3208b04-40f3-4f1d-b4d5-0f92ce9774db None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:37.897802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3208b04-40f3-4f1d-b4d5-0f92ce9774db None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:37.897802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3208b04-40f3-4f1d-b4d5-0f92ce9774db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:37.916758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e3208b04-40f3-4f1d-b4d5-0f92ce9774db None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:37.917950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e3208b04-40f3-4f1d-b4d5-0f92ce9774db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:37.926725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 741/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:37.964325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6501afc-9f01-421d-b59f-faadc5e8a3d4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:37.979126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6501afc-9f01-421d-b59f-faadc5e8a3d4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:37.983084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619be6b-65c5-4415-91df-d4307bd7eb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:37.983725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619be6b-65c5-4415-91df-d4307bd7eb8c service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:37.984004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619be6b-65c5-4415-91df-d4307bd7eb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:37.984274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619be6b-65c5-4415-91df-d4307bd7eb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:37.995088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619be6b-65c5-4415-91df-d4307bd7eb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:37.996769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619be6b-65c5-4415-91df-d4307bd7eb8c service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:38.008319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 714/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:40.135110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f30b-9356-4e5b-837b-9193c7a45a72 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:40.135644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f30b-9356-4e5b-837b-9193c7a45a72 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:40.135644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f30b-9356-4e5b-837b-9193c7a45a72 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:40.137059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 742/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:40.143089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b66c9-4e99-4bbf-9261-47b8c90d29d5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:40.143551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b66c9-4e99-4bbf-9261-47b8c90d29d5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:40.143645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b66c9-4e99-4bbf-9261-47b8c90d29d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:40.161177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9b3b66c9-4e99-4bbf-9261-47b8c90d29d5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:40.162387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9b3b66c9-4e99-4bbf-9261-47b8c90d29d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:40.170203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 715/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:40.210079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f30b-9356-4e5b-837b-9193c7a45a72 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:40.221704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f30b-9356-4e5b-837b-9193c7a45a72 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:40.225322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1393d-0605-476a-b7c9-09d81227f352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:40.225967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1393d-0605-476a-b7c9-09d81227f352 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:40.226245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1393d-0605-476a-b7c9-09d81227f352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:40.226548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1393d-0605-476a-b7c9-09d81227f352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:40.236038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1393d-0605-476a-b7c9-09d81227f352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:40.237562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1393d-0605-476a-b7c9-09d81227f352 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:40.247231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 743/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:42.380423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4b98a-a734-4738-9e60-cc33da0db560 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:42.380969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4b98a-a734-4738-9e60-cc33da0db560 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:42.380969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4b98a-a734-4738-9e60-cc33da0db560 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:42.382315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 716/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:42.387716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699338a9-a64b-42f5-9170-0e3436fb586f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:42.388200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699338a9-a64b-42f5-9170-0e3436fb586f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:42.388296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699338a9-a64b-42f5-9170-0e3436fb586f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:42.406020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-699338a9-a64b-42f5-9170-0e3436fb586f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:42.407236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-699338a9-a64b-42f5-9170-0e3436fb586f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:42.415737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 744/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:42.453996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4b98a-a734-4738-9e60-cc33da0db560 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:42.465145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4b98a-a734-4738-9e60-cc33da0db560 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:42.468676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061b39a-e873-4c52-9ae5-b13af1b1b18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:42.469324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061b39a-e873-4c52-9ae5-b13af1b1b18e service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:42.469609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061b39a-e873-4c52-9ae5-b13af1b1b18e service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:42.469919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061b39a-e873-4c52-9ae5-b13af1b1b18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:42.479889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061b39a-e873-4c52-9ae5-b13af1b1b18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:42.481363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061b39a-e873-4c52-9ae5-b13af1b1b18e service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:42.491702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 717/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:44.601573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6422e-86a2-4645-b4cd-bbb8477f2157 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:44.602156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6422e-86a2-4645-b4cd-bbb8477f2157 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:44.602156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6422e-86a2-4645-b4cd-bbb8477f2157 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:44.603481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 745/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:44.608934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83321ab-e185-4833-a6c3-5491ad981393 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:44.609401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83321ab-e185-4833-a6c3-5491ad981393 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:44.609493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83321ab-e185-4833-a6c3-5491ad981393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:44.616116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=79 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:12:44.629023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e83321ab-e185-4833-a6c3-5491ad981393 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:44.630227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e83321ab-e185-4833-a6c3-5491ad981393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:44.638014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 718/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:44.676779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6422e-86a2-4645-b4cd-bbb8477f2157 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:44.682141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=73 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:12:44.689214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6422e-86a2-4645-b4cd-bbb8477f2157 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:44.692730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c32b0-1fc0-4d5d-b58c-ae186c6f2460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:44.693372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c32b0-1fc0-4d5d-b58c-ae186c6f2460 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:44.693655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c32b0-1fc0-4d5d-b58c-ae186c6f2460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:44.693930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c32b0-1fc0-4d5d-b58c-ae186c6f2460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:44.703325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c32b0-1fc0-4d5d-b58c-ae186c6f2460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:44.704782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c32b0-1fc0-4d5d-b58c-ae186c6f2460 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:44.714466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 746/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:46.820061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc15dab-a817-4348-980f-1cce6ba535a8 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:46.820623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc15dab-a817-4348-980f-1cce6ba535a8 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:46.821060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc15dab-a817-4348-980f-1cce6ba535a8 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:46.823381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 719/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:46.828898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117648e9-cebb-43d2-83e8-5478425e80e0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:46.829420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117648e9-cebb-43d2-83e8-5478425e80e0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:46.829557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117648e9-cebb-43d2-83e8-5478425e80e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:46.847984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-117648e9-cebb-43d2-83e8-5478425e80e0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:46.849270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-117648e9-cebb-43d2-83e8-5478425e80e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:46.857110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 747/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:46.894755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc15dab-a817-4348-980f-1cce6ba535a8 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:46.908892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc15dab-a817-4348-980f-1cce6ba535a8 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:46.913201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6679812-6730-4a60-94be-1fddec96546f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:46.913854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679812-6730-4a60-94be-1fddec96546f service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:46.914137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679812-6730-4a60-94be-1fddec96546f service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:46.914415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679812-6730-4a60-94be-1fddec96546f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:46.923927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6679812-6730-4a60-94be-1fddec96546f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:46.925528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6679812-6730-4a60-94be-1fddec96546f service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:46.935802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 720/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:12:49.016399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302759e-a731-4c49-99b3-040a984b4b6a None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:49.016989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302759e-a731-4c49-99b3-040a984b4b6a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:49.017087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302759e-a731-4c49-99b3-040a984b4b6a None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:49.018442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 748/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:12:49.024226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b84be7-60d4-46ce-b536-9e3d1e7490da None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:49.024764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b84be7-60d4-46ce-b536-9e3d1e7490da None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:49.024857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b84be7-60d4-46ce-b536-9e3d1e7490da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:49.043318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c3b84be7-60d4-46ce-b536-9e3d1e7490da None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:12:49.044533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c3b84be7-60d4-46ce-b536-9e3d1e7490da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:12:49.053040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 721/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:12:49.091331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302759e-a731-4c49-99b3-040a984b4b6a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:12:49.105605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302759e-a731-4c49-99b3-040a984b4b6a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:12:49.109088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0817ec3-40f1-49c0-9605-005d6e9b1a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:12:49.109730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0817ec3-40f1-49c0-9605-005d6e9b1a93 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:12:49.110009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0817ec3-40f1-49c0-9605-005d6e9b1a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:12:49.110284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0817ec3-40f1-49c0-9605-005d6e9b1a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:12:49.120121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0817ec3-40f1-49c0-9605-005d6e9b1a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:12:49.121576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0817ec3-40f1-49c0-9605-005d6e9b1a93 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:12:49.132183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 749/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:12:59.037780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=11 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:12:59.126966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=14 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:13:09.317872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eec353-4eae-4942-93ff-39c95357e38f None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:09.318570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eec353-4eae-4942-93ff-39c95357e38f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:09.318570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eec353-4eae-4942-93ff-39c95357e38f None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:09.319775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 722/1471] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:09.325434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d675a5a-8fa6-41ed-a4c4-f3388024c2ee None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:09.325841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d675a5a-8fa6-41ed-a4c4-f3388024c2ee None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:09.325941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d675a5a-8fa6-41ed-a4c4-f3388024c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:09.344439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6d675a5a-8fa6-41ed-a4c4-f3388024c2ee None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:09.345668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6d675a5a-8fa6-41ed-a4c4-f3388024c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:09.353481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 750/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:13:09.401955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873f07-bbec-4710-9dbc-c70c559bd0f9 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:09.402240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873f07-bbec-4710-9dbc-c70c559bd0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:09.402517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873f07-bbec-4710-9dbc-c70c559bd0f9 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:09.403899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 723/1473] 10.222.0.52 () {54 vars in 962 bytes} [Fri Jun 24 09:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:09.409285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93b87-e90d-4b03-b83e-212c910b3788 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:09.409576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93b87-e90d-4b03-b83e-212c910b3788 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:09.409882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93b87-e90d-4b03-b83e-212c910b3788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:09.428801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-20b93b87-e90d-4b03-b83e-212c910b3788 None None] MFA Rules not processed for user `28a07f47b8954418a5a86f344fb7fbe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:09.429991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-20b93b87-e90d-4b03-b83e-212c910b3788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:09.436741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 751/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Fri Jun 24 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:13:09.444169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03873f07-bbec-4710-9dbc-c70c559bd0f9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:13:09.455005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03873f07-bbec-4710-9dbc-c70c559bd0f9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:13:09.458458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a40d23-efbc-487f-8fab-07978759340b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:13:09.459092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40d23-efbc-487f-8fab-07978759340b service glance] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:09.459370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40d23-efbc-487f-8fab-07978759340b service glance] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:09.459644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40d23-efbc-487f-8fab-07978759340b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:09.470076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a40d23-efbc-487f-8fab-07978759340b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:13:09.471538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a40d23-efbc-487f-8fab-07978759340b service glance] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:13:09.481184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 724/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:13:11.661369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5857-e277-4cf5-b0e5-fcf5c148d8ce None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:11.661923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5857-e277-4cf5-b0e5-fcf5c148d8ce None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:11.661923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5857-e277-4cf5-b0e5-fcf5c148d8ce None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:11.663190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 752/1476] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:11.668496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6691-999a-4585-841f-1b737d2cd97a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:11.668957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6691-999a-4585-841f-1b737d2cd97a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:11.669066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6691-999a-4585-841f-1b737d2cd97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:11.688090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-aa2a6691-999a-4585-841f-1b737d2cd97a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:11.689337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-aa2a6691-999a-4585-841f-1b737d2cd97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:11.697156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 725/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:13:11.721599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5857-e277-4cf5-b0e5-fcf5c148d8ce None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:13:11.734553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5857-e277-4cf5-b0e5-fcf5c148d8ce None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:13:11.739114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53622495-fa0f-4408-8f0d-14c7ca8f10c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:13:11.739767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622495-fa0f-4408-8f0d-14c7ca8f10c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:11.740042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622495-fa0f-4408-8f0d-14c7ca8f10c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:11.740315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622495-fa0f-4408-8f0d-14c7ca8f10c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:11.741076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53622495-fa0f-4408-8f0d-14c7ca8f10c7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:13:11.742855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-53622495-fa0f-4408-8f0d-14c7ca8f10c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:13:11.742855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:13:11.742855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:13:11.743990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53622495-fa0f-4408-8f0d-14c7ca8f10c7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:13:11.765219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 753/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Jun 24 09:13:11 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:13:13.862385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc43479-11e8-4d65-ad8d-bf59a86fb4d2 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:13.862958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc43479-11e8-4d65-ad8d-bf59a86fb4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:13.862958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc43479-11e8-4d65-ad8d-bf59a86fb4d2 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:13.864323 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 726/1479] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:13.870312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4729eb-015b-4046-b2bb-e34feb90ca75 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:13.870780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4729eb-015b-4046-b2bb-e34feb90ca75 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:13.870870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4729eb-015b-4046-b2bb-e34feb90ca75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:13.891977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ac4729eb-015b-4046-b2bb-e34feb90ca75 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:13.893380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ac4729eb-015b-4046-b2bb-e34feb90ca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:13.901437 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 754/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:13:13.946091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc43479-11e8-4d65-ad8d-bf59a86fb4d2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:13:13.960843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc43479-11e8-4d65-ad8d-bf59a86fb4d2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:13:13.966212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d61a7-ed9d-4e64-93b0-9d09c8c658a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:13:13.966860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d61a7-ed9d-4e64-93b0-9d09c8c658a7 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:13.967143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d61a7-ed9d-4e64-93b0-9d09c8c658a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:13.967423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d61a7-ed9d-4e64-93b0-9d09c8c658a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:13.979441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d61a7-ed9d-4e64-93b0-9d09c8c658a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:13:13.982062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d61a7-ed9d-4e64-93b0-9d09c8c658a7 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:13:13.992587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 727/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:13:16.254498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5da1a2-c055-4d0c-b2cb-3088babaa805 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:16.255326 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5da1a2-c055-4d0c-b2cb-3088babaa805 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:16.255326 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5da1a2-c055-4d0c-b2cb-3088babaa805 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:16.256409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 755/1482] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:16.262049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6261e5-6031-4c15-968e-8814e8664fd2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:16.262507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6261e5-6031-4c15-968e-8814e8664fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:16.262640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6261e5-6031-4c15-968e-8814e8664fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:16.281135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7b6261e5-6031-4c15-968e-8814e8664fd2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:16.282348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7b6261e5-6031-4c15-968e-8814e8664fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:16.290677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 728/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:13:16.328547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5da1a2-c055-4d0c-b2cb-3088babaa805 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:13:16.339344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5da1a2-c055-4d0c-b2cb-3088babaa805 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:13:16.342790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e0c53-1a03-4bb4-9437-12069a094ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:13:16.343454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e0c53-1a03-4bb4-9437-12069a094ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:16.343761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e0c53-1a03-4bb4-9437-12069a094ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:16.344049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e0c53-1a03-4bb4-9437-12069a094ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:16.353285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e0c53-1a03-4bb4-9437-12069a094ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:13:16.354695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e0c53-1a03-4bb4-9437-12069a094ed9 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:13:16.364048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 756/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:13:18.475147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3893-b020-4c82-a41e-28f7246d568e None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:18.475721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3893-b020-4c82-a41e-28f7246d568e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:18.475721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3893-b020-4c82-a41e-28f7246d568e None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:18.477096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 729/1485] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:18.482320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6d37-57b3-4d2a-95dd-3c68189a16e5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:18.482775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6d37-57b3-4d2a-95dd-3c68189a16e5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:18.482874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6d37-57b3-4d2a-95dd-3c68189a16e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:18.500942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ab6c6d37-57b3-4d2a-95dd-3c68189a16e5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:18.502239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ab6c6d37-57b3-4d2a-95dd-3c68189a16e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:18.510282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 757/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:13:18.548790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b3893-b020-4c82-a41e-28f7246d568e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:13:18.559921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b3893-b020-4c82-a41e-28f7246d568e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:13:18.563479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd3efd-9fa4-4cba-ab47-4b90efb13a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:13:18.564153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3efd-9fa4-4cba-ab47-4b90efb13a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:18.564430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3efd-9fa4-4cba-ab47-4b90efb13a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:18.564744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3efd-9fa4-4cba-ab47-4b90efb13a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:18.574176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd3efd-9fa4-4cba-ab47-4b90efb13a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:13:18.575636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd3efd-9fa4-4cba-ab47-4b90efb13a5e service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:13:18.585192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 730/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:13:20.885586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2d4f2-d5ce-4e0f-afb6-bb0897176515 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:20.886177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2d4f2-d5ce-4e0f-afb6-bb0897176515 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:20.886177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2d4f2-d5ce-4e0f-afb6-bb0897176515 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:20.887508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 758/1488] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:20.893518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d142dc9-b566-4e59-bc80-ace40fa753e7 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:20.894022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d142dc9-b566-4e59-bc80-ace40fa753e7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:20.894135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d142dc9-b566-4e59-bc80-ace40fa753e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:20.913083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3d142dc9-b566-4e59-bc80-ace40fa753e7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:20.914289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3d142dc9-b566-4e59-bc80-ace40fa753e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:20.922827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 731/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:13:20.963736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2d4f2-d5ce-4e0f-afb6-bb0897176515 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:13:20.980026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2d4f2-d5ce-4e0f-afb6-bb0897176515 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:13:20.983559 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d6dcb-e041-40cd-aefb-450913c62968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:13:20.984234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d6dcb-e041-40cd-aefb-450913c62968 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:20.984516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d6dcb-e041-40cd-aefb-450913c62968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:20.984825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d6dcb-e041-40cd-aefb-450913c62968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:20.994225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d6dcb-e041-40cd-aefb-450913c62968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:13:20.995671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d6dcb-e041-40cd-aefb-450913c62968 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:13:21.007768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 759/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:13:23.295428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0758-a310-4452-9aa7-4f6c629f139b None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:23.295990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0758-a310-4452-9aa7-4f6c629f139b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:23.295990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea0758-a310-4452-9aa7-4f6c629f139b None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:23.297366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 732/1491] 10.222.0.52 () {54 vars in 926 bytes} [Fri Jun 24 09:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:13:23.302786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2a417-e2ff-4cfc-887b-40eb501b7a35 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:23.303245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2a417-e2ff-4cfc-887b-40eb501b7a35 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:23.303347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2a417-e2ff-4cfc-887b-40eb501b7a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:23.321974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f8c2a417-e2ff-4cfc-887b-40eb501b7a35 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:13:23.323169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f8c2a417-e2ff-4cfc-887b-40eb501b7a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:13:23.332389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 760/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Fri Jun 24 09:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:13:23.371418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0758-a310-4452-9aa7-4f6c629f139b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:13:23.383530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea0758-a310-4452-9aa7-4f6c629f139b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:13:23.387096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52376f-614f-4279-ab34-032cd25b8daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:13:23.387748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52376f-614f-4279-ab34-032cd25b8daf service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:13:23.388029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52376f-614f-4279-ab34-032cd25b8daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:13:23.388305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52376f-614f-4279-ab34-032cd25b8daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:13:23.398544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52376f-614f-4279-ab34-032cd25b8daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:13:23.399991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52376f-614f-4279-ab34-032cd25b8daf service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:13:23.410548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 733/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:13:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:13:33.316513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=33 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:13:33.405417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=33 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:15:06.834111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324c0e-aeb7-4292-9b2b-ef8a8ddfdd96 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:06.834682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324c0e-aeb7-4292-9b2b-ef8a8ddfdd96 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:06.834682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324c0e-aeb7-4292-9b2b-ef8a8ddfdd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:06.854536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-84324c0e-aeb7-4292-9b2b-ef8a8ddfdd96 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:15:06.855739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-84324c0e-aeb7-4292-9b2b-ef8a8ddfdd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:15:06.863668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 761/1494] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:15:06.874017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173addc-47b1-4687-ac44-a1913c216299 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:06.874581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173addc-47b1-4687-ac44-a1913c216299 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:06.874581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173addc-47b1-4687-ac44-a1913c216299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:06.892652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-2173addc-47b1-4687-ac44-a1913c216299 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:15:06.893878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2173addc-47b1-4687-ac44-a1913c216299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:15:06.901701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 734/1495] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:15:06.906364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84324c0e-aeb7-4292-9b2b-ef8a8ddfdd96 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:06.916650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84324c0e-aeb7-4292-9b2b-ef8a8ddfdd96 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:06.921557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:06.922229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:06.922514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:06.922793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:06.923522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:15:06.923831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:06.924886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:06.924886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:15:06.924886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:15:06.925812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:06.939416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 762/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:15:06.948090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173addc-47b1-4687-ac44-a1913c216299 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:06.957440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173addc-47b1-4687-ac44-a1913c216299 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:06.962025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:06.962686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:06.962975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:06.963817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:06.964526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:06.966456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING py.warnings [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:06.966456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: warnings.warn(msg) Jun 24 09:15:06.966456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: Jun 24 09:15:06.967680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.003310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 735/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:15:06 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:15:07.008434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:07.020171 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44293cb4-4adc-4d50-90d0-988dd30b7367 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:07.078216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:07.078895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.079180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.079480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.080165 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:07.081205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:07.081205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:15:07.081205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:15:07.082401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.105174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 763/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:15:07 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:15:07.109343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:07.120236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445335-89a7-4e8a-805d-b63f599d587e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:07.124823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:07.125492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.125773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.126049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.126772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:07.127783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING py.warnings [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:07.127783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: warnings.warn(msg) Jun 24 09:15:07.127783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: Jun 24 09:15:07.128679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.142571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 736/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:15:07.146834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:07.156948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05a9bb-be16-41b5-b1cf-ec3f2e7984b2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:07.161380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1655048-1fc9-47b4-9564-3cbe62a70e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:07.162035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655048-1fc9-47b4-9564-3cbe62a70e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.162319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655048-1fc9-47b4-9564-3cbe62a70e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.162726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655048-1fc9-47b4-9564-3cbe62a70e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.163459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1655048-1fc9-47b4-9564-3cbe62a70e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:07.164461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-f1655048-1fc9-47b4-9564-3cbe62a70e7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:07.164461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:15:07.164461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:15:07.165369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1655048-1fc9-47b4-9564-3cbe62a70e7f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.179967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 764/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:15:07.185144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:07.195333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88259a40-dc25-48ea-a3c9-c9fa161f105c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:07.200518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0127f3b-837e-4ca3-81cb-efefaaf6ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:07.201431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0127f3b-837e-4ca3-81cb-efefaaf6ae25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.201714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0127f3b-837e-4ca3-81cb-efefaaf6ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.201993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0127f3b-837e-4ca3-81cb-efefaaf6ae25 admin admin] PATH_INFO: `/v3/projects/93aaf14e99c94ee78ca283f189bafafd/users/4d814027521c4b6f8df14ccecdfdcf61/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.217281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0127f3b-837e-4ca3-81cb-efefaaf6ae25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93aaf14e99c94ee78ca283f189bafafd, user_id=4d814027521c4b6f8df14ccecdfdcf61, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:07.218340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING py.warnings [None req-c0127f3b-837e-4ca3-81cb-efefaaf6ae25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:07.218340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: warnings.warn(msg) Jun 24 09:15:07.218340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: Jun 24 09:15:07.220235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0127f3b-837e-4ca3-81cb-efefaaf6ae25 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.233762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 737/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:15:07 2022] PUT /identity/v3/projects/93aaf14e99c94ee78ca283f189bafafd/users/4d814027521c4b6f8df14ccecdfdcf61/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:15:07.245711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07b2c-1d18-4649-92de-099816cdb5b9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.245999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07b2c-1d18-4649-92de-099816cdb5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.246271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07b2c-1d18-4649-92de-099816cdb5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.273321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-73f07b2c-1d18-4649-92de-099816cdb5b9 None None] MFA Rules not processed for user `4d814027521c4b6f8df14ccecdfdcf61`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:15:07.274527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-73f07b2c-1d18-4649-92de-099816cdb5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:15:07.339828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 765/1502] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:15:07.352854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462478e-e322-4d47-a87e-bc151bf936b3 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.353143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462478e-e322-4d47-a87e-bc151bf936b3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.353418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462478e-e322-4d47-a87e-bc151bf936b3 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.355468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 738/1503] 10.222.0.52 () {54 vars in 963 bytes} [Fri Jun 24 09:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:15:07.360474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cb400-b65c-4e87-8075-e9b6ae9414c0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.360864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cb400-b65c-4e87-8075-e9b6ae9414c0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.361172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cb400-b65c-4e87-8075-e9b6ae9414c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.379297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0c4cb400-b65c-4e87-8075-e9b6ae9414c0 None None] MFA Rules not processed for user `31bf0e50e20b47d0a63662bbecce833d`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:15:07.380493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0c4cb400-b65c-4e87-8075-e9b6ae9414c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:15:07.426916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:15:07.436936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 766/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Fri Jun 24 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:15:07.444627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462478e-e322-4d47-a87e-bc151bf936b3 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:07.455846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462478e-e322-4d47-a87e-bc151bf936b3 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:07.459355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:07.459991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.460270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.460549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.470588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:07.472017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.481011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 739/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:15:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:15:07.523313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2c184-f0e8-4d44-b710-7b7b8cf6d37c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.523603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2c184-f0e8-4d44-b710-7b7b8cf6d37c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.523908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2c184-f0e8-4d44-b710-7b7b8cf6d37c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.525207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 767/1506] 10.222.0.52 () {52 vars in 829 bytes} [Fri Jun 24 09:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:15:07.529476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:07.597950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251203a-a67c-4fd9-b39b-347329f70df8 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:07.603369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:07.604825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.605109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.605387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] PATH_INFO: `/v3/users/4d814027521c4b6f8df14ccecdfdcf61` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.606667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d814027521c4b6f8df14ccecdfdcf61)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:07.607647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING py.warnings [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:07.607647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: warnings.warn(msg) Jun 24 09:15:07.607647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: Jun 24 09:15:07.608886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.678859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d814027521c4b6f8df14ccecdfdcf61'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:07.689326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d814027521c4b6f8df14ccecdfdcf61'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:07.701759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d814027521c4b6f8df14ccecdfdcf61'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:07.702572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d814027521c4b6f8df14ccecdfdcf61'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:07.703198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d814027521c4b6f8df14ccecdfdcf61'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:07.719580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:15:07.729330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 740/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:15:07 2022] DELETE /identity/v3/users/4d814027521c4b6f8df14ccecdfdcf61 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:15:07.733780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.734086 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.734367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.753089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:15:07.754317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:15:07.817697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 768/1508] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:15:07.832530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:07.834176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:15:07.895764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0f2e6-7df8-4efb-91b5-2d251e4a9d27 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:07.900180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc3d91-53d3-4387-8884-8d0b7ef0cfea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:07.901458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc3d91-53d3-4387-8884-8d0b7ef0cfea service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:07.901735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc3d91-53d3-4387-8884-8d0b7ef0cfea service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:07.902007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc3d91-53d3-4387-8884-8d0b7ef0cfea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:07.912278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc3d91-53d3-4387-8884-8d0b7ef0cfea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:07.913780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc3d91-53d3-4387-8884-8d0b7ef0cfea service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:07.924586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 741/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:15:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:15:08.268602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:15:08.270222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:15:08.286375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bb325-4bbc-47e0-b16e-f7d1c84cc65d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:15:08.291754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:15:08.292922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:15:08.292922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:15:08.293154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] PATH_INFO: `/v3/projects/93aaf14e99c94ee78ca283f189bafafd` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:15:08.294352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93aaf14e99c94ee78ca283f189bafafd)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:15:08.295985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:15:08.295985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:15:08.295985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:15:08.297246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:15:08.361668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93aaf14e99c94ee78ca283f189bafafd', 'request_id': 'req-7ed4f673-dfac-48b2-b639-2c5e617e8941'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:08.372096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93aaf14e99c94ee78ca283f189bafafd', 'request_id': 'req-7ed4f673-dfac-48b2-b639-2c5e617e8941'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:08.378631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93aaf14e99c94ee78ca283f189bafafd', 'request_id': 'req-7ed4f673-dfac-48b2-b639-2c5e617e8941'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:08.398853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] The token cache is being invalidate because project 93aaf14e99c94ee78ca283f189bafafd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:15:08.399136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:08.399816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7ed4f673-dfac-48b2-b639-2c5e617e8941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:15:08.402116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 769/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:15:08 2022] DELETE /identity/v3/projects/93aaf14e99c94ee78ca283f189bafafd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:15:17.920090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:15:18.395746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:16:49.958556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b120d0-90b9-4af7-8254-dcda352c1610 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:49.959181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b120d0-90b9-4af7-8254-dcda352c1610 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:49.959181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b120d0-90b9-4af7-8254-dcda352c1610 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:49.960436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 742/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:16:49.965566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ce43b-3f4a-44bf-b3a6-963ca55b3951 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:49.965979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ce43b-3f4a-44bf-b3a6-963ca55b3951 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:49.966131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ce43b-3f4a-44bf-b3a6-963ca55b3951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:49.985718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-066ce43b-3f4a-44bf-b3a6-963ca55b3951 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:16:49.986918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-066ce43b-3f4a-44bf-b3a6-963ca55b3951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:16:50.045452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 770/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:16:50.198820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b120d0-90b9-4af7-8254-dcda352c1610 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:16:50.200515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b2b120d0-90b9-4af7-8254-dcda352c1610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:16:50.276885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b120d0-90b9-4af7-8254-dcda352c1610 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:16:50.282137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1d32-5d1a-4571-8f73-b57a34765380 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:16:50.282781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1d32-5d1a-4571-8f73-b57a34765380 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:50.283059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1d32-5d1a-4571-8f73-b57a34765380 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:50.283334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1d32-5d1a-4571-8f73-b57a34765380 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:50.293275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e1d32-5d1a-4571-8f73-b57a34765380 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:16:50.293697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e1d32-5d1a-4571-8f73-b57a34765380 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:16:50.303645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 743/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:16:51.135323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5db5f-d763-43e9-baa4-a790f0e35642 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:51.135873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5db5f-d763-43e9-baa4-a790f0e35642 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:51.135873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5db5f-d763-43e9-baa4-a790f0e35642 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:51.137279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 771/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:16:51.142081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbadba-2ef1-493d-951b-3a24d75cf0d1 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:51.142522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbadba-2ef1-493d-951b-3a24d75cf0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:51.142647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbadba-2ef1-493d-951b-3a24d75cf0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:51.162524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-95dbadba-2ef1-493d-951b-3a24d75cf0d1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:16:51.163729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-95dbadba-2ef1-493d-951b-3a24d75cf0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:16:51.172474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 744/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:16:51.328360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5db5f-d763-43e9-baa4-a790f0e35642 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:16:51.342575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5db5f-d763-43e9-baa4-a790f0e35642 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:16:51.347977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d8fa5-72fd-4d5f-b61e-c18e647550ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:16:51.348647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d8fa5-72fd-4d5f-b61e-c18e647550ab service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:51.348922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d8fa5-72fd-4d5f-b61e-c18e647550ab service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:51.349196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d8fa5-72fd-4d5f-b61e-c18e647550ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:51.357815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d8fa5-72fd-4d5f-b61e-c18e647550ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:16:51.359385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d8fa5-72fd-4d5f-b61e-c18e647550ab service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:16:51.369428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 772/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:16:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:16:52.297081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f559ff-5787-4397-80d8-971821baa405 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:52.297587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f559ff-5787-4397-80d8-971821baa405 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:52.297587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f559ff-5787-4397-80d8-971821baa405 None None] PATH_INFO: `/v3` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:52.298868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 745/1517] 10.222.0.59 () {54 vars in 925 bytes} [Fri Jun 24 09:16:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:16:52.305337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9e8a-f879-4c1b-a976-a297cf6eb966 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:52.305783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9e8a-f879-4c1b-a976-a297cf6eb966 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:52.305895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9e8a-f879-4c1b-a976-a297cf6eb966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:52.324144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5a6e9e8a-f879-4c1b-a976-a297cf6eb966 None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:16:52.325369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5a6e9e8a-f879-4c1b-a976-a297cf6eb966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:16:52.386445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 773/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Jun 24 09:16:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:16:56.345786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06bc7e-29e8-4e8f-a3d8-b4f6feda0ad9 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:56.346310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06bc7e-29e8-4e8f-a3d8-b4f6feda0ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:56.346310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06bc7e-29e8-4e8f-a3d8-b4f6feda0ad9 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:56.347638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 746/1519] 10.222.0.52 () {54 vars in 960 bytes} [Fri Jun 24 09:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:16:56.352721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9e667-dd03-46bb-9ea9-e7295247bece None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:56.353153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9e667-dd03-46bb-9ea9-e7295247bece None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:56.353292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9e667-dd03-46bb-9ea9-e7295247bece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:56.371546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6af9e667-dd03-46bb-9ea9-e7295247bece None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:16:56.372757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6af9e667-dd03-46bb-9ea9-e7295247bece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:16:56.381174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 774/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Fri Jun 24 09:16:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:16:56.389289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06bc7e-29e8-4e8f-a3d8-b4f6feda0ad9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:16:56.402173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06bc7e-29e8-4e8f-a3d8-b4f6feda0ad9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:16:56.406933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e4ba0-1e93-4887-9e06-9d3d9a3f0f40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:16:56.407549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e4ba0-1e93-4887-9e06-9d3d9a3f0f40 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:16:56.407824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e4ba0-1e93-4887-9e06-9d3d9a3f0f40 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:16:56.408090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e4ba0-1e93-4887-9e06-9d3d9a3f0f40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:16:56.416260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e4ba0-1e93-4887-9e06-9d3d9a3f0f40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:16:56.417925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e4ba0-1e93-4887-9e06-9d3d9a3f0f40 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:16:56.427347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 747/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:16:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:17:02.603374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d328e-c967-4f1d-9a02-d9d9139cc7dc None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:02.603949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d328e-c967-4f1d-9a02-d9d9139cc7dc None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:02.603949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d328e-c967-4f1d-9a02-d9d9139cc7dc None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:02.605309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 775/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:17:02.610167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d854dd-180d-42a5-b418-78e1bb0bd664 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:02.610605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d854dd-180d-42a5-b418-78e1bb0bd664 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:02.610701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d854dd-180d-42a5-b418-78e1bb0bd664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:02.628094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e8d854dd-180d-42a5-b418-78e1bb0bd664 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:17:02.629304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e8d854dd-180d-42a5-b418-78e1bb0bd664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:02.637649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 748/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:17:02.791807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d328e-c967-4f1d-9a02-d9d9139cc7dc None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:17:02.806581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d328e-c967-4f1d-9a02-d9d9139cc7dc None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:17:02.811914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c531eb-8022-42b6-ba88-0b83f861528a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:17:02.812545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c531eb-8022-42b6-ba88-0b83f861528a service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:02.812851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c531eb-8022-42b6-ba88-0b83f861528a service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:02.813126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c531eb-8022-42b6-ba88-0b83f861528a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:02.821794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c531eb-8022-42b6-ba88-0b83f861528a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:17:02.823373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c531eb-8022-42b6-ba88-0b83f861528a service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:17:02.832879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 776/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:17:03.647600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e75d6-8ef9-4d36-9fbc-e36f56c53239 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:03.648157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e75d6-8ef9-4d36-9fbc-e36f56c53239 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:03.648157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e75d6-8ef9-4d36-9fbc-e36f56c53239 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:03.649559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 749/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:17:03.654428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8a20-005b-4de3-a05c-9a6eef5b4a1f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:03.654834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8a20-005b-4de3-a05c-9a6eef5b4a1f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:03.654993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e8a20-005b-4de3-a05c-9a6eef5b4a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:03.673227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-966e8a20-005b-4de3-a05c-9a6eef5b4a1f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:17:03.674409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-966e8a20-005b-4de3-a05c-9a6eef5b4a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:03.683368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 777/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:17:03.833721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e75d6-8ef9-4d36-9fbc-e36f56c53239 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:17:03.848462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e75d6-8ef9-4d36-9fbc-e36f56c53239 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:17:03.854639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a0ec3-5669-4d96-b3b2-4d9a1ef110a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:17:03.855300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0ec3-5669-4d96-b3b2-4d9a1ef110a0 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:03.855593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0ec3-5669-4d96-b3b2-4d9a1ef110a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:03.855862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0ec3-5669-4d96-b3b2-4d9a1ef110a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:03.864449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a0ec3-5669-4d96-b3b2-4d9a1ef110a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:17:03.866099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a0ec3-5669-4d96-b3b2-4d9a1ef110a0 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:17:03.875421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 750/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:17:07.889961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e3b27-c250-4469-af9e-9927581308a7 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:07.890533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e3b27-c250-4469-af9e-9927581308a7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:07.890533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e3b27-c250-4469-af9e-9927581308a7 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:07.891781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 778/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:17:07.897173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769fccb-d059-4eea-b7d6-d0088066fb26 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:07.897558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769fccb-d059-4eea-b7d6-d0088066fb26 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:07.897828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769fccb-d059-4eea-b7d6-d0088066fb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:07.915826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7769fccb-d059-4eea-b7d6-d0088066fb26 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:17:07.917046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7769fccb-d059-4eea-b7d6-d0088066fb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:07.924782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 751/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:17:07.961200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e3b27-c250-4469-af9e-9927581308a7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:17:07.962884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d24e3b27-c250-4469-af9e-9927581308a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:08.019886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e3b27-c250-4469-af9e-9927581308a7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:17:08.023484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43013d-9946-4dbb-8fc7-0107a44ff6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:17:08.024140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43013d-9946-4dbb-8fc7-0107a44ff6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:08.024427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43013d-9946-4dbb-8fc7-0107a44ff6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:08.024738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43013d-9946-4dbb-8fc7-0107a44ff6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:08.033893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43013d-9946-4dbb-8fc7-0107a44ff6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:17:08.035320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43013d-9946-4dbb-8fc7-0107a44ff6e2 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:17:08.040957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:17:08.046566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 779/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:17:10.082088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f817-1852-4adf-ab90-3575bd4e1d3e None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:10.082631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f817-1852-4adf-ab90-3575bd4e1d3e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:10.082631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f817-1852-4adf-ab90-3575bd4e1d3e None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:10.084060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 752/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:17:10.089181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a5e6d-9fd8-4bfa-a00a-6bc9f5ff327e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:10.089572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a5e6d-9fd8-4bfa-a00a-6bc9f5ff327e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:10.089742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a5e6d-9fd8-4bfa-a00a-6bc9f5ff327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:10.107939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5b8a5e6d-9fd8-4bfa-a00a-6bc9f5ff327e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:17:10.109443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5b8a5e6d-9fd8-4bfa-a00a-6bc9f5ff327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:10.117782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 780/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:17:10.152815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763f817-1852-4adf-ab90-3575bd4e1d3e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:17:10.164104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763f817-1852-4adf-ab90-3575bd4e1d3e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:17:10.167648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc318-8619-4470-9ddb-862b24f31e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:17:10.168414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc318-8619-4470-9ddb-862b24f31e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:10.168778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc318-8619-4470-9ddb-862b24f31e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:10.169060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc318-8619-4470-9ddb-862b24f31e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:10.178860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc318-8619-4470-9ddb-862b24f31e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:17:10.180535 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc318-8619-4470-9ddb-862b24f31e4f service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:17:10.189808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 753/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:17:13.721269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a03272-999c-4636-8a06-082f9e3ac22c None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:13.721828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a03272-999c-4636-8a06-082f9e3ac22c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:13.721828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a03272-999c-4636-8a06-082f9e3ac22c None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:13.723188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 781/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:17:13.728943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25952f-3bed-46b3-98b4-86d0bda21169 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:13.729383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25952f-3bed-46b3-98b4-86d0bda21169 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:13.729498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25952f-3bed-46b3-98b4-86d0bda21169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:13.747916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8a25952f-3bed-46b3-98b4-86d0bda21169 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:17:13.749087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8a25952f-3bed-46b3-98b4-86d0bda21169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:13.756898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 754/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:17:14.068160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a03272-999c-4636-8a06-082f9e3ac22c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:17:14.069912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4a03272-999c-4636-8a06-082f9e3ac22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:14.128525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a03272-999c-4636-8a06-082f9e3ac22c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:17:14.132055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f7b77-b80f-4b54-814c-474c8871c3ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:17:14.132776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7b77-b80f-4b54-814c-474c8871c3ac service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:14.133020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7b77-b80f-4b54-814c-474c8871c3ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:14.133290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f7b77-b80f-4b54-814c-474c8871c3ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:14.142098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f7b77-b80f-4b54-814c-474c8871c3ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:17:14.143507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f7b77-b80f-4b54-814c-474c8871c3ac service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:17:14.153699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 782/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:17:16.234818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93707-3fc4-4bb9-ad4f-8fb1a13dc749 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:16.235348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93707-3fc4-4bb9-ad4f-8fb1a13dc749 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:16.235348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93707-3fc4-4bb9-ad4f-8fb1a13dc749 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:16.236781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 755/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:17:16.242379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca734a4-e39f-4b42-af8c-dc17e8ceb918 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:16.242799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca734a4-e39f-4b42-af8c-dc17e8ceb918 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:16.242933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca734a4-e39f-4b42-af8c-dc17e8ceb918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:16.261030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bca734a4-e39f-4b42-af8c-dc17e8ceb918 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:17:16.262232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bca734a4-e39f-4b42-af8c-dc17e8ceb918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:16.270297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 783/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:17:16.574264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c93707-3fc4-4bb9-ad4f-8fb1a13dc749 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:17:16.575953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c9c93707-3fc4-4bb9-ad4f-8fb1a13dc749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:17:16.590260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c93707-3fc4-4bb9-ad4f-8fb1a13dc749 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:17:16.594640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac9855-ed8a-40b7-957e-d78d80d08d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:17:16.595401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9855-ed8a-40b7-957e-d78d80d08d0a service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:17:16.595594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9855-ed8a-40b7-957e-d78d80d08d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:17:16.595965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9855-ed8a-40b7-957e-d78d80d08d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:17:16.605749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac9855-ed8a-40b7-957e-d78d80d08d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:17:16.607220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac9855-ed8a-40b7-957e-d78d80d08d0a service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:17:16.616958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 756/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:17:26.255979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=32 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:17:26.612525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=70 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:47.292363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a854e69-0dbe-4eba-860f-d0498e8103c1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.292363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a854e69-0dbe-4eba-860f-d0498e8103c1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.292363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a854e69-0dbe-4eba-860f-d0498e8103c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.316322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1a854e69-0dbe-4eba-860f-d0498e8103c1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.317808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1a854e69-0dbe-4eba-860f-d0498e8103c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.331340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 784/1540] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:47.343349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479566-5b8c-4882-9bf9-e12f836c9b05 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.343927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479566-5b8c-4882-9bf9-e12f836c9b05 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.343927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479566-5b8c-4882-9bf9-e12f836c9b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.370532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ae183-a298-4c9a-8372-1d349c45363c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.370876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ae183-a298-4c9a-8372-1d349c45363c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.371083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ae183-a298-4c9a-8372-1d349c45363c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.374493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c4479566-5b8c-4882-9bf9-e12f836c9b05 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.375829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c4479566-5b8c-4882-9bf9-e12f836c9b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.396889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5b8ae183-a298-4c9a-8372-1d349c45363c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.398075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 757/1541] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:47.398212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5b8ae183-a298-4c9a-8372-1d349c45363c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.408387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479566-5b8c-4882-9bf9-e12f836c9b05 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:47.411653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 785/1542] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:18:47.422662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd8617-b262-4f30-b2ce-424b29c1451a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.423014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd8617-b262-4f30-b2ce-424b29c1451a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.423204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd8617-b262-4f30-b2ce-424b29c1451a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.425153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479566-5b8c-4882-9bf9-e12f836c9b05 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:47.438017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:47.438743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.439027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.440665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.441507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:47.441822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:47.442948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING py.warnings [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:18:47.442948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: warnings.warn(msg) Jun 24 09:18:47.442948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: Jun 24 09:18:47.443850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:47.464667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 758/1543] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:18:47.465415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-72dd8617-b262-4f30-b2ce-424b29c1451a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.467340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-72dd8617-b262-4f30-b2ce-424b29c1451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.484008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:47.494312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 786/1544] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:18:47.498399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a73b7-9c96-498e-b737-ed8c03a1a397 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:47.499663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd8617-b262-4f30-b2ce-424b29c1451a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:47.510578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:47.511270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.511557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.511837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.512588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:47.515256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:47.516576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd8617-b262-4f30-b2ce-424b29c1451a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:47.529094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:47.529747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.530034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.530310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.531057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:47.531362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:47.533031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:47.572038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 787/1545] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:18:47.577718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:47.578776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 759/1546] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:18:47.580862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:47.590936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca51f47-4ea7-4eb0-b780-89b9bd01f1d3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:47.690922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c428-30bf-48e0-8405-9b0e6a3b1d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:47.691630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c428-30bf-48e0-8405-9b0e6a3b1d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.691913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c428-30bf-48e0-8405-9b0e6a3b1d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.692190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c428-30bf-48e0-8405-9b0e6a3b1d10 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.696918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c428-30bf-48e0-8405-9b0e6a3b1d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:47.699465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c428-30bf-48e0-8405-9b0e6a3b1d10 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:47.741329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 788/1547] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:47.742845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e3dcb-975b-402c-a88b-c774e5c7f436 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.743129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e3dcb-975b-402c-a88b-c774e5c7f436 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.743401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e3dcb-975b-402c-a88b-c774e5c7f436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.750154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88002a4-5104-4d71-a6fc-3374b947d8f9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:47.755315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceab942-6119-4bc8-828e-8782e42d5975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:47.755994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab942-6119-4bc8-828e-8782e42d5975 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.756275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab942-6119-4bc8-828e-8782e42d5975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.756570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab942-6119-4bc8-828e-8782e42d5975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.757275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceab942-6119-4bc8-828e-8782e42d5975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:47.759374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceab942-6119-4bc8-828e-8782e42d5975 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:47.764634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7a6e3dcb-975b-402c-a88b-c774e5c7f436 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.765882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7a6e3dcb-975b-402c-a88b-c774e5c7f436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.775906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 789/1548] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:47.777311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5f6f9-3d88-4a36-8fb1-1d3399695b41 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.777885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5f6f9-3d88-4a36-8fb1-1d3399695b41 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.778217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5f6f9-3d88-4a36-8fb1-1d3399695b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.796761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 760/1549] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/projects => generated 375 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:47.798282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916b45-a6d8-4cb1-aaf3-517a99ba5579 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.798567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916b45-a6d8-4cb1-aaf3-517a99ba5579 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.798843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916b45-a6d8-4cb1-aaf3-517a99ba5579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.810993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5fb5f6f9-3d88-4a36-8fb1-1d3399695b41 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.811704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5fb5f6f9-3d88-4a36-8fb1-1d3399695b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.850919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ce916b45-a6d8-4cb1-aaf3-517a99ba5579 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.852039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ce916b45-a6d8-4cb1-aaf3-517a99ba5579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.892610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 790/1550] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:47.892610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6962f-feff-4ea8-a704-27ac2fb50943 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.892610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6962f-feff-4ea8-a704-27ac2fb50943 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.892610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6962f-feff-4ea8-a704-27ac2fb50943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.913125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-76f6962f-feff-4ea8-a704-27ac2fb50943 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.914633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-76f6962f-feff-4ea8-a704-27ac2fb50943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.923513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 791/1551] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:47.924267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6962f-feff-4ea8-a704-27ac2fb50943 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:47.928517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 761/1552] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:47.930206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de90d6-9400-4307-8b05-19a0c7b0483f None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.930505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de90d6-9400-4307-8b05-19a0c7b0483f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.930789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de90d6-9400-4307-8b05-19a0c7b0483f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.937104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6962f-feff-4ea8-a704-27ac2fb50943 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:47.944548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:47.945256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.945855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.946448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.947838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:47.949633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:47.955332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-b6de90d6-9400-4307-8b05-19a0c7b0483f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.957148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b6de90d6-9400-4307-8b05-19a0c7b0483f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:47.961387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 792/1553] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:47.962112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:47.965576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 762/1554] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:47.966649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84bda-0ead-4b31-b916-8f5d93cb2fa3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.966937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84bda-0ead-4b31-b916-8f5d93cb2fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.967216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84bda-0ead-4b31-b916-8f5d93cb2fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.973919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef400-d45d-42b7-8150-184b7e00a576 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:47.980997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492795f-cade-4629-bbb5-f6c4becde0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:47.981675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492795f-cade-4629-bbb5-f6c4becde0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:47.981958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492795f-cade-4629-bbb5-f6c4becde0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:47.982236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492795f-cade-4629-bbb5-f6c4becde0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:47.982932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492795f-cade-4629-bbb5-f6c4becde0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:47.984897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492795f-cade-4629-bbb5-f6c4becde0e9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:47.990552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e5f84bda-0ead-4b31-b916-8f5d93cb2fa3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:47.991767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e5f84bda-0ead-4b31-b916-8f5d93cb2fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.000349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 763/1555] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.001774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1b3b-d30f-43ab-a167-3c7321223f13 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.001986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1b3b-d30f-43ab-a167-3c7321223f13 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.002360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1b3b-d30f-43ab-a167-3c7321223f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.008239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 793/1556] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:47 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.009545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe6ce1-8911-4cb9-8740-27125320f0d3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.009919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe6ce1-8911-4cb9-8740-27125320f0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.010205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe6ce1-8911-4cb9-8740-27125320f0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.026401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1fbb1b3b-d30f-43ab-a167-3c7321223f13 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.027609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1fbb1b3b-d30f-43ab-a167-3c7321223f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.028933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3efe6ce1-8911-4cb9-8740-27125320f0d3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.030101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3efe6ce1-8911-4cb9-8740-27125320f0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.037794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 764/1557] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.043680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7c5f0-7f73-4fe3-ad52-3340cef51021 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.043680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7c5f0-7f73-4fe3-ad52-3340cef51021 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.043680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7c5f0-7f73-4fe3-ad52-3340cef51021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.046849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 794/1558] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.048438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42858b-2bd3-4ab5-bfed-e31e65a98423 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.048699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42858b-2bd3-4ab5-bfed-e31e65a98423 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.048980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42858b-2bd3-4ab5-bfed-e31e65a98423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.064241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-bad7c5f0-7f73-4fe3-ad52-3340cef51021 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.065494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bad7c5f0-7f73-4fe3-ad52-3340cef51021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.068471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ea42858b-2bd3-4ab5-bfed-e31e65a98423 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.069677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ea42858b-2bd3-4ab5-bfed-e31e65a98423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.074365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 765/1559] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.075128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7c5f0-7f73-4fe3-ad52-3340cef51021 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.081016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 795/1560] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.081016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42858b-2bd3-4ab5-bfed-e31e65a98423 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.086473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:48.086890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7c5f0-7f73-4fe3-ad52-3340cef51021 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.091924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42858b-2bd3-4ab5-bfed-e31e65a98423 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.093530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eade4ff-8416-413d-a369-012f5ebb8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.094504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eade4ff-8416-413d-a369-012f5ebb8d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.094794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eade4ff-8416-413d-a369-012f5ebb8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.095104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eade4ff-8416-413d-a369-012f5ebb8d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.096008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eade4ff-8416-413d-a369-012f5ebb8d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.097687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.097969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eade4ff-8416-413d-a369-012f5ebb8d1f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.098612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.098904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.099200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.099973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:48.100576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.102032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.109166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 766/1561] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:48.110549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724f34f-af1f-47ea-9411-ba90637efbd4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.110835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724f34f-af1f-47ea-9411-ba90637efbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.111108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724f34f-af1f-47ea-9411-ba90637efbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.117739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 796/1562] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:18:48.118578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.129311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc341ed8-1e8b-4e06-82a7-c104f4794eb5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.130343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:48.134536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6275d9-1604-4bf2-9dc6-1f728af52fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.135278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6275d9-1604-4bf2-9dc6-1f728af52fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.135574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6275d9-1604-4bf2-9dc6-1f728af52fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.135855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6275d9-1604-4bf2-9dc6-1f728af52fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.136613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6275d9-1604-4bf2-9dc6-1f728af52fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.136883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7724f34f-af1f-47ea-9411-ba90637efbd4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.138239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6275d9-1604-4bf2-9dc6-1f728af52fb5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.138542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7724f34f-af1f-47ea-9411-ba90637efbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.146603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 767/1563] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.147490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724f34f-af1f-47ea-9411-ba90637efbd4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.149449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 797/1564] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:48.150588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c5764-1502-4f17-b092-e3868ca214eb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.150879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c5764-1502-4f17-b092-e3868ca214eb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.151162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c5764-1502-4f17-b092-e3868ca214eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.157488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724f34f-af1f-47ea-9411-ba90637efbd4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.162141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.162831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.163250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.163536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.164261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:48.164592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.166198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.168495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-798c5764-1502-4f17-b092-e3868ca214eb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.169747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-798c5764-1502-4f17-b092-e3868ca214eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.178402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 798/1565] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.182734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 768/1566] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:18:48.182851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c5764-1502-4f17-b092-e3868ca214eb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.182943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.190057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363940de-fb62-4e1f-9510-7a761683ab74 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.196121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.196850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.197128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.197401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.198155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:48.198353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c5764-1502-4f17-b092-e3868ca214eb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.198467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.200050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.204189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.205663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.206131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.206131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.207215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:48.207576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.209213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.213196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 769/1567] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:18:48.213813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.222840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 799/1568] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:18:48.223636 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.233923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb4cde-dffc-4ad7-b41d-de0709cd849a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.234536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07353f-8ad9-48d0-a4ea-b0699bb84a5f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.239211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.239861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.240078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.240200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.240401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.241002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.241113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.241285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.241559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] PATH_INFO: `/v3/projects/67221e1503c34e3d9fa3f13cd40c37dc/users/f8333f6a9dab4618bc0e6b2060f15250/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.242115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:18:48.242115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:18:48.242115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:18:48.243284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.256850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67221e1503c34e3d9fa3f13cd40c37dc, user_id=f8333f6a9dab4618bc0e6b2060f15250, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.259713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.270650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 800/1569] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.271704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.275342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 770/1570] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:48 2022] PUT /identity/v3/projects/67221e1503c34e3d9fa3f13cd40c37dc/users/f8333f6a9dab4618bc0e6b2060f15250/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:48.276727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.336908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b1575-1820-420e-b127-ff9dcd38c606 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.338563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b25aa-35c5-4597-98a4-7f45ae6cb0c6 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.341545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.342215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.342496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.342768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.343284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.343491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.343947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.344373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.344498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.345111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.345398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:48.345723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.347844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.355700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 801/1571] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:48.357031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.360297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 771/1572] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:18:48.361068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.371859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f04b7-bee4-4e5a-94cb-afc8724ff377 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.372806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cbe10-c20e-4608-bd7f-a5c0ec08a24e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.376832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.377305 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.377492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.377777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.377961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.378059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.378243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.378603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.378731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.379315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:18:48.379606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.380632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.381213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.394061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 802/1573] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:18:48.394887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.405733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b0dc0-9e9f-4d44-8f72-e275931de2f3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.410833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.411395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 772/1574] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.412294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.412706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.413090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.413428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.414141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.416227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.443923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 803/1575] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.444778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.478528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011efe0-49fb-4994-a4f2-7053b728c771 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.511028 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:48.511462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:48.515546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bf256-d785-4e38-a541-a6d1f8d39683 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.521652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbc25e-8d2b-455a-8769-277ba0fb1785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.522643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbc25e-8d2b-455a-8769-277ba0fb1785 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.522914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbc25e-8d2b-455a-8769-277ba0fb1785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.523189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbc25e-8d2b-455a-8769-277ba0fb1785 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.523910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbc25e-8d2b-455a-8769-277ba0fb1785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.525876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbc25e-8d2b-455a-8769-277ba0fb1785 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.546927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.547603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.547883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.548157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.548898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.551425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.552497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 804/1576] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.555324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d3619-f369-4797-9298-ad51da015596 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.555828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d3619-f369-4797-9298-ad51da015596 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.556333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d3619-f369-4797-9298-ad51da015596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.583878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 773/1577] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.583878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.585302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-871d3619-f369-4797-9298-ad51da015596 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.590313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-871d3619-f369-4797-9298-ad51da015596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.652249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18824477-ea97-4032-917f-ee77ea0c798f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.656317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 805/1578] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.657266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d3619-f369-4797-9298-ad51da015596 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.661147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.662784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.663405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.663915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] PATH_INFO: `/v3/projects/eec4b7af6b284140be130acf01e1710f/users/6ea7013dd9424b9db1798e81521ee903/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.670415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d3619-f369-4797-9298-ad51da015596 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.675117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.676376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.676696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.677001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.677699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.679907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.683062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec4b7af6b284140be130acf01e1710f, user_id=6ea7013dd9424b9db1798e81521ee903, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.685764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.700156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 774/1579] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:48 2022] PUT /identity/v3/projects/eec4b7af6b284140be130acf01e1710f/users/6ea7013dd9424b9db1798e81521ee903/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:48.701000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.706972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 806/1580] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.707777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.767907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193adf1-1785-45f3-a9a9-9d974687541c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.771837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab1f62-2b59-4aee-bac9-564594804aee admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.773267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.773945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.774235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.774519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.775220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.776981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.777293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.777656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.777937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.778221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.778929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.780864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.803743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 807/1581] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/users => generated 304 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.804851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.807542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 775/1582] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.808473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.815609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd122d57-4472-4fc9-9596-64aa53c146ab admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.834265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:48.869357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:48.874751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ebf7-a2a1-4972-b6e7-5dd58ee2ecf9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.878035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.878861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.879247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.879633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.879748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.880310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.880647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.880897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.881003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.881629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.883089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.883197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.893586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 776/1583] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:48.895092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.905950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a8f61-4efa-4ab4-a56f-a5a40c6fb97b admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.911661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 808/1584] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.911661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.911661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.914649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16bbf1-0574-40e4-b2ad-2a8d289603d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.915331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16bbf1-0574-40e4-b2ad-2a8d289603d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.915618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16bbf1-0574-40e4-b2ad-2a8d289603d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.915900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16bbf1-0574-40e4-b2ad-2a8d289603d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.916617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16bbf1-0574-40e4-b2ad-2a8d289603d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.917643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING py.warnings [None req-0c16bbf1-0574-40e4-b2ad-2a8d289603d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:18:48.917643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: warnings.warn(msg) Jun 24 09:18:48.917643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: Jun 24 09:18:48.918862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16bbf1-0574-40e4-b2ad-2a8d289603d5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.939608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 777/1585] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:48.940960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d081f21-b6e5-4021-bdbf-698d2ae7ef69 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.941252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d081f21-b6e5-4021-bdbf-698d2ae7ef69 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.941538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d081f21-b6e5-4021-bdbf-698d2ae7ef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.961324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-4d081f21-b6e5-4021-bdbf-698d2ae7ef69 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:48.962547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4d081f21-b6e5-4021-bdbf-698d2ae7ef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:48.970231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 778/1586] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:48.974326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d081f21-b6e5-4021-bdbf-698d2ae7ef69 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:48.976815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeedc86-b138-4281-b6a7-11460c55f536 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.980929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.981370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d081f21-b6e5-4021-bdbf-698d2ae7ef69 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:48.981582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.981863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.982136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.985890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:48.986549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:48.986830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:48.987104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:48.987847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.989790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:48.991661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:48.993119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.002973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 809/1587] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:49.004256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.010498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 779/1588] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:48 2022] POST /identity/v3/users => generated 297 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:49.014353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.015682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d5c31-5dd4-4508-a845-5de805230264 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.022795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.023465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.024073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.024217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.024921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.026546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.029992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032deec-f752-4ff1-baa7-cfb011efd5c8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.037008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.037855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.038221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.038867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 810/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.038998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.039722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.039831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.042285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.050415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a157f70-4f4f-4180-b9fd-d81825ba8fb9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.055947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.056716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.057004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.057317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.058107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.060036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.065628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 780/1590] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:49 2022] POST /identity/v3/users => generated 302 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:49.066565 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.077862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 811/1591] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.077862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.078155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61023b-18c6-4144-be05-0c45c4f40c56 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.083011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.083698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.083984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.084263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.085212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.086836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.089495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709571-1315-4de2-a9e9-b361623d6489 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.094982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.095691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.095929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.096200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.096997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.099259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.099735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 781/1592] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.103525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.114481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 812/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.114481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.115151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42a39e-5985-4c18-8a68-da2c9db7438f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.120067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.120923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.121261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.121540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.125011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10c83a-7e7c-4117-8e8d-2209f8c0ccd8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.130648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.131316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.131597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.131871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.131974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.132625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.133409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.134222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.144982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 782/1594] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:49.145834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.146432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 813/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.150108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.156596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2e5f8-96cb-4d0c-934d-a8513a03e62d service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.160106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fcfa2-ea38-47a5-9765-1836ef8793f9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.161636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.162302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.162593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.162877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.163606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.165277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.165840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.166517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.166801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.167077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.167802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.169431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.177538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 783/1596] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.178328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.184356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 814/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.185088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.189122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b766b8d-e003-4808-8135-8c8b2f3078dd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.196237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.197408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.197699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.197977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] PATH_INFO: `/v3/projects/89cd37f7227940b59dbcceb9525c9704/users/00556257371a41b39bce3ca8ee545238/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.202462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96cc0d-5426-4b75-baf5-680f9f4f9dec admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.208613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.209254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.209543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.209814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.210544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.212148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.217986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cd37f7227940b59dbcceb9525c9704, user_id=00556257371a41b39bce3ca8ee545238, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.221213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.224598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 815/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.224598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.234604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c36725-8d2d-4506-9dff-b15aaf5268a4 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.241823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 784/1599] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:49 2022] PUT /identity/v3/projects/89cd37f7227940b59dbcceb9525c9704/users/00556257371a41b39bce3ca8ee545238/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:49.242635 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.315475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.317099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.317591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.318129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.319099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.321104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.325543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf2f5c-e6af-4ddb-b237-971656e298ef admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.333220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.333922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.334205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.334549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.334700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 816/1600] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.335289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.335787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.336934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.338449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:49.346817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12a622-d465-46be-bb3e-db0499207e4c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.351878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 785/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.352801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.353287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a7905-f708-49fa-aa50-42981cede00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.354412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7905-f708-49fa-aa50-42981cede00c admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.354757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7905-f708-49fa-aa50-42981cede00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.355045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7905-f708-49fa-aa50-42981cede00c admin admin] PATH_INFO: `/v3/projects/836bfadb581e4e2fb7a5514f8ef6fff9/users/6063674fe3064e41b89e91533c3e0258/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.363319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47074027-629c-4c23-b505-3962dc2e4ace admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.370982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.371896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.372184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.372458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.373567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.375178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.376477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a7905-f708-49fa-aa50-42981cede00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=836bfadb581e4e2fb7a5514f8ef6fff9, user_id=6063674fe3064e41b89e91533c3e0258, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.377850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-2e2a7905-f708-49fa-aa50-42981cede00c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:18:49.377850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:18:49.377850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:18:49.385144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a7905-f708-49fa-aa50-42981cede00c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.387321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:49.392873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 786/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:49.393763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.404312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 817/1603] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:49 2022] PUT /identity/v3/projects/836bfadb581e4e2fb7a5514f8ef6fff9/users/6063674fe3064e41b89e91533c3e0258/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:49.405360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055355e-2811-4501-86d2-70a6184897dd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.405660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b23649-d2d7-4c29-b400-a68a34c0b099 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.405950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b23649-d2d7-4c29-b400-a68a34c0b099 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.406352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b23649-d2d7-4c29-b400-a68a34c0b099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.424299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-81b23649-d2d7-4c29-b400-a68a34c0b099 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:49.426293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-81b23649-d2d7-4c29-b400-a68a34c0b099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:49.485325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.486334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.486714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.487074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] PATH_INFO: `/v3/projects/93da016997ef4f3798af64e5093cd238/users/074ce31d6a8c4d3caaee8b7d1b74af77/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.510309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93da016997ef4f3798af64e5093cd238, user_id=074ce31d6a8c4d3caaee8b7d1b74af77, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.514638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.544732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 818/1604] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:49.544732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b23649-d2d7-4c29-b400-a68a34c0b099 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.558499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b23649-d2d7-4c29-b400-a68a34c0b099 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.563826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 787/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:49 2022] PUT /identity/v3/projects/93da016997ef4f3798af64e5093cd238/users/074ce31d6a8c4d3caaee8b7d1b74af77/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:49.565382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.668092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.669059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.669349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.669629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] PATH_INFO: `/v3/projects/5c235d441df94f80a393e5639413d4d9/users/592836cc56ed4334bb46890a74460307/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.687826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5a738-10fd-4b3b-8954-773f1d590923 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.689482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c235d441df94f80a393e5639413d4d9, user_id=592836cc56ed4334bb46890a74460307, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.693038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.694114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b43a67-67af-48ee-ae45-e797d7929b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.695129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43a67-67af-48ee-ae45-e797d7929b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.695478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43a67-67af-48ee-ae45-e797d7929b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.695760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43a67-67af-48ee-ae45-e797d7929b58 admin admin] PATH_INFO: `/v3/projects/ff102b486fa846498ea8013ca6bcc63c/users/b97b26dd3bb5427a8edfe4eb653b945b/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.711925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 819/1606] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:49 2022] PUT /identity/v3/projects/5c235d441df94f80a393e5639413d4d9/users/592836cc56ed4334bb46890a74460307/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:49.712892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.718814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b43a67-67af-48ee-ae45-e797d7929b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff102b486fa846498ea8013ca6bcc63c, user_id=b97b26dd3bb5427a8edfe4eb653b945b, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.721562 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b43a67-67af-48ee-ae45-e797d7929b58 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.746713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 788/1607] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:49 2022] PUT /identity/v3/projects/ff102b486fa846498ea8013ca6bcc63c/users/b97b26dd3bb5427a8edfe4eb653b945b/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:49.752977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae1c25-143e-4032-a9dc-ca100a55028e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.754289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae1c25-143e-4032-a9dc-ca100a55028e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.754621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae1c25-143e-4032-a9dc-ca100a55028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.806075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31cf90-06bb-4976-8818-e8ff0881977a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.816964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-83ae1c25-143e-4032-a9dc-ca100a55028e None None] MFA Rules not processed for user `6063674fe3064e41b89e91533c3e0258`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:49.818832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-83ae1c25-143e-4032-a9dc-ca100a55028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:49.833378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:49.871239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:49.884244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1837e3-41be-4698-904c-7ef60ee00469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:49.885375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1837e3-41be-4698-904c-7ef60ee00469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.885658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1837e3-41be-4698-904c-7ef60ee00469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.886083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1837e3-41be-4698-904c-7ef60ee00469 admin admin] PATH_INFO: `/v3/projects/bb248b98ec2e4011b059436cef046c14/users/291e03c93abe444a88a3103573a082d0/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.902154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 789/1608] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:49.905675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae1c25-143e-4032-a9dc-ca100a55028e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:49.906286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1837e3-41be-4698-904c-7ef60ee00469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb248b98ec2e4011b059436cef046c14, user_id=291e03c93abe444a88a3103573a082d0, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:49.909223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1837e3-41be-4698-904c-7ef60ee00469 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:49.929206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 820/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:49 2022] PUT /identity/v3/projects/bb248b98ec2e4011b059436cef046c14/users/291e03c93abe444a88a3103573a082d0/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:49.931771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7b1e4-79db-4347-b183-86a385e77a87 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:49.932153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7b1e4-79db-4347-b183-86a385e77a87 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:49.932512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7b1e4-79db-4347-b183-86a385e77a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:49.965072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae1c25-143e-4032-a9dc-ca100a55028e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:49.965831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8ee7b1e4-79db-4347-b183-86a385e77a87 None None] MFA Rules not processed for user `074ce31d6a8c4d3caaee8b7d1b74af77`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:49.973652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8ee7b1e4-79db-4347-b183-86a385e77a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:50.046222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.047831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.048961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.049163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.078254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 821/1610] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:50.080275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0ad94-1593-4da2-a6da-62a934736486 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.080600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0ad94-1593-4da2-a6da-62a934736486 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.080964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0ad94-1593-4da2-a6da-62a934736486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.106714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5de0ad94-1593-4da2-a6da-62a934736486 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:50.108866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5de0ad94-1593-4da2-a6da-62a934736486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:50.210275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 822/1611] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:50.212992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8993d-6701-4d2e-a617-c206abdc09c7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.213288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8993d-6701-4d2e-a617-c206abdc09c7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.213571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8993d-6701-4d2e-a617-c206abdc09c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.227541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.240345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.263185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 790/1612] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:50.263185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.270210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7ae8993d-6701-4d2e-a617-c206abdc09c7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:50.271500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7ae8993d-6701-4d2e-a617-c206abdc09c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:50.279545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1351f65-e8b6-4409-8416-6c7403dfbb1e service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.281610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 823/1613] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:50.283018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbda16-366b-442f-b46b-a45e318c28ab None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.283410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbda16-366b-442f-b46b-a45e318c28ab None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.283836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbda16-366b-442f-b46b-a45e318c28ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.293782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.294224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.294517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.294801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.302038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.306213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.309373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-88fbda16-366b-442f-b46b-a45e318c28ab None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:50.312197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-88fbda16-366b-442f-b46b-a45e318c28ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:50.325930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 824/1614] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:50.327485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbda16-366b-442f-b46b-a45e318c28ab None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.348062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbda16-366b-442f-b46b-a45e318c28ab None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.355723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 791/1615] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:50 2022] POST /identity/v3/projects => generated 407 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:50.355723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.358452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.359768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.360549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.361017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.362990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.366309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.411761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 825/1616] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:50 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:50.413359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.419132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:50.423056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:50.487240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb558d-8e12-42b7-b087-4d783dae73e8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.524451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc529dc1-120b-476e-a158-a99f6bcaf9d5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.539382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.540079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.540366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.540672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.595534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.596493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.597027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.597981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.625600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.627426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.641175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 826/1617] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:50.642226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.653061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacf9c4-a642-4885-888d-d89f46bc5bd5 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.657995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.658777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.659377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.659791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.672811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.674895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.691890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 827/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:50.693150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.709804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a131c-c1c8-455b-9828-1e2d642eb4e9 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.716746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.717522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.717884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.718237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.719392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.721758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.735007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.736784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.749515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 828/1619] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:50 2022] POST /identity/v3/users => generated 310 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:50.750485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.764261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 792/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:50.765474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.770213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e380a3d-4689-4f62-8174-7d77cfe360b6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.780597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.781471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.781903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.782258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.783225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.786865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.789041 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9f58d-2920-4c7c-b424-129fbac331fd service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.803752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.805492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.806168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.806629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.807604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.811710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.821633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 829/1621] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:50 2022] POST /identity/v3/users => generated 293 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:50.822769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.829529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 793/1622] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:50.835606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.848481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d91a-4850-4b57-89cb-d12b454a3799 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.860955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.861658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.861949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.862234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.863126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.864846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.889071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9cd54-5134-4610-a457-af54525b1834 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.898860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.900477 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.901978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.902625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] PATH_INFO: `/v3/projects/0cfd69768cd04e509e2aca6d77b24129/users/4b026e302a954d8eb9d6c2c3a2857368/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.904173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 830/1623] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:50.905116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:50.930149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102658c5-7e21-45a8-9757-cd5e76021608 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:50.935714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cfd69768cd04e509e2aca6d77b24129, user_id=4b026e302a954d8eb9d6c2c3a2857368, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:50.939760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:50.947315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:50.949254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:50.949726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:50.950045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] PATH_INFO: `/v3/projects/38ab3c8bad3e4e53b066a9cd25037a7b/users/152b364b885446f0af724255621def53/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:50.978152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 794/1624] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:50 2022] PUT /identity/v3/projects/0cfd69768cd04e509e2aca6d77b24129/users/4b026e302a954d8eb9d6c2c3a2857368/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:50.987527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:51.006411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ab3c8bad3e4e53b066a9cd25037a7b, user_id=152b364b885446f0af724255621def53, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:51.015724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:51.072775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 831/1625] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:50 2022] PUT /identity/v3/projects/38ab3c8bad3e4e53b066a9cd25037a7b/users/152b364b885446f0af724255621def53/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:51.077206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:51.173335 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:51.197996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:51.209598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd652-5320-431b-a880-10be55c7f116 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:51.271024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29237ec9-9eae-43ea-88d2-b4b608076e3c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:51.289190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:51.290300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.290597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.290876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.291827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:51.293484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:51.312234 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:51.313216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.313658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.314075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 832/1626] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:51.314404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.315469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:51.318543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:51.318852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:51.338325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 795/1627] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:51.339542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:51.344026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185bdfc-6c90-4dc8-a478-19fa31352057 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:51.350829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8541d-a54d-4504-a50f-e67d54b872e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:51.351880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8541d-a54d-4504-a50f-e67d54b872e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.352014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8541d-a54d-4504-a50f-e67d54b872e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.352451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8541d-a54d-4504-a50f-e67d54b872e8 admin admin] PATH_INFO: `/v3/projects/38ab3c8bad3e4e53b066a9cd25037a7b/users/152b364b885446f0af724255621def53/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.354556 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8541d-a54d-4504-a50f-e67d54b872e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ab3c8bad3e4e53b066a9cd25037a7b, user_id=152b364b885446f0af724255621def53, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:51.357779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8541d-a54d-4504-a50f-e67d54b872e8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:51.370988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35095ff-2bc7-4689-9810-d96e60a32f2e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:51.374803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 833/1628] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:51 2022] PUT /identity/v3/projects/38ab3c8bad3e4e53b066a9cd25037a7b/users/152b364b885446f0af724255621def53/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:51.386945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684c5da-159a-4333-8d70-10dbc798cc19 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.387428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684c5da-159a-4333-8d70-10dbc798cc19 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.387815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684c5da-159a-4333-8d70-10dbc798cc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.447428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebbf13-5612-4721-8fd5-478a639269d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:51.448406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebbf13-5612-4721-8fd5-478a639269d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.448683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebbf13-5612-4721-8fd5-478a639269d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.448966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebbf13-5612-4721-8fd5-478a639269d5 admin admin] PATH_INFO: `/v3/projects/0cfd69768cd04e509e2aca6d77b24129/users/4b026e302a954d8eb9d6c2c3a2857368/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.450686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7684c5da-159a-4333-8d70-10dbc798cc19 None None] MFA Rules not processed for user `152b364b885446f0af724255621def53`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:51.451043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebbf13-5612-4721-8fd5-478a639269d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cfd69768cd04e509e2aca6d77b24129, user_id=4b026e302a954d8eb9d6c2c3a2857368, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:51.453986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebbf13-5612-4721-8fd5-478a639269d5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:51.456490 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7684c5da-159a-4333-8d70-10dbc798cc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:51.466329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 796/1629] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:51 2022] PUT /identity/v3/projects/0cfd69768cd04e509e2aca6d77b24129/users/4b026e302a954d8eb9d6c2c3a2857368/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:51.480830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce88c-7781-403e-adaf-80b7ff353829 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.481297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce88c-7781-403e-adaf-80b7ff353829 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.481657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ce88c-7781-403e-adaf-80b7ff353829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.526271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f57ce88c-7781-403e-adaf-80b7ff353829 None None] MFA Rules not processed for user `4b026e302a954d8eb9d6c2c3a2857368`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:51.527719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f57ce88c-7781-403e-adaf-80b7ff353829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:51.631819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 797/1630] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:51 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:51.645970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce88c-7781-403e-adaf-80b7ff353829 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:51.654687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 834/1631] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:18:51.656453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684c5da-159a-4333-8d70-10dbc798cc19 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:51.696211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:51.749713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ce88c-7781-403e-adaf-80b7ff353829 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:51.754490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:51.755426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.755856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.756225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.766320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684c5da-159a-4333-8d70-10dbc798cc19 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:51.777147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:51.777704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:51.779023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:51.779317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:51.779437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:51.781657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:51.792882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 798/1632] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:51 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:51.914254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:51.937552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:51.939807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:51.973630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 835/1633] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:52.231330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:52.231330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388cc1a-0458-471c-9941-02764fb0b944 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:52.231330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985825d5-9c2c-4fa4-907f-5c664198dd59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:52.231330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985825d5-9c2c-4fa4-907f-5c664198dd59 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:52.233375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985825d5-9c2c-4fa4-907f-5c664198dd59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:52.233375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985825d5-9c2c-4fa4-907f-5c664198dd59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:52.395010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985825d5-9c2c-4fa4-907f-5c664198dd59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:52.397765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985825d5-9c2c-4fa4-907f-5c664198dd59 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:52.437543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 799/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:52 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:52.936188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:53.113802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7960ca5-ecff-458b-82f2-320d8b83c631 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:53.128746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:53.130431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:53.131536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:53.132026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] PATH_INFO: `/v3/users/074ce31d6a8c4d3caaee8b7d1b74af77` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:53.134554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074ce31d6a8c4d3caaee8b7d1b74af77)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:53.138127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: WARNING py.warnings [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:18:53.138127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: warnings.warn(msg) Jun 24 09:18:53.138127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: Jun 24 09:18:53.143416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:53.309803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074ce31d6a8c4d3caaee8b7d1b74af77'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:53.320626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074ce31d6a8c4d3caaee8b7d1b74af77'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:53.347035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074ce31d6a8c4d3caaee8b7d1b74af77'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:53.349156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074ce31d6a8c4d3caaee8b7d1b74af77'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:53.350214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074ce31d6a8c4d3caaee8b7d1b74af77'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:53.458892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 836/1635] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:18:52 2022] DELETE /identity/v3/users/074ce31d6a8c4d3caaee8b7d1b74af77 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:53.465089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cbf0-2885-464b-bcd4-10198b0a2c1b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:53.466093 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cbf0-2885-464b-bcd4-10198b0a2c1b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:53.466581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cbf0-2885-464b-bcd4-10198b0a2c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:53.502624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5ef1cbf0-2885-464b-bcd4-10198b0a2c1b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:53.506259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5ef1cbf0-2885-464b-bcd4-10198b0a2c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:53.715967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 800/1636] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:18:53.789117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:53.790985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:53.957559 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844074-b9be-4d44-b56b-43c2ff31b2e7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:53.980138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:53.980875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:53.982450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:53.982849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:54.008334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:54.014754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:54.045650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 837/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:18:55.262923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878764-a32a-4236-a561-69990cc86aed None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:55.264786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878764-a32a-4236-a561-69990cc86aed None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:55.265782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878764-a32a-4236-a561-69990cc86aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:55.324615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-03878764-a32a-4236-a561-69990cc86aed None None] MFA Rules not processed for user `291e03c93abe444a88a3103573a082d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:55.329021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-03878764-a32a-4236-a561-69990cc86aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:55.439270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:55.469475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 801/1638] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:18:55.474662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:55.477492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:55.514808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f45f83-64c1-4187-8af3-67362aee8b4c service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:55.528920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:55.532284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:55.532833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:55.533947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:55.535781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:55.542008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:55.553437 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:55.598302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 838/1639] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:18:55 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:55.603355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03878764-a32a-4236-a561-69990cc86aed None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:55.625558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03878764-a32a-4236-a561-69990cc86aed None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:55.873163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:55.876106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:55.876844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:55.877315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:55.878875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:55.880774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:55.881085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:55.886877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:55.919341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e1ffa-ff80-43b5-bc92-8b4db1d35e8b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:55.938720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:55.946564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:55.946904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:55.951603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] PATH_INFO: `/v3/projects/93da016997ef4f3798af64e5093cd238` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:55.955203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=93da016997ef4f3798af64e5093cd238)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:55.955692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 802/1640] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:18:55 2022] POST /identity/v3/users => generated 309 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:18:55.957664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:55.959840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:55.990064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77540d94-7951-45ee-a8fd-9a2ebb08970a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:56.019997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:56.020748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:56.021035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:56.021311 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:56.025992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:56.031060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:56.063945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 803/1641] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:56.083658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:56.100495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93da016997ef4f3798af64e5093cd238', 'request_id': 'req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.106126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd6bb4-d623-4764-aedf-5e58abf4a483 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:56.119673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93da016997ef4f3798af64e5093cd238', 'request_id': 'req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.127807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93da016997ef4f3798af64e5093cd238', 'request_id': 'req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.164867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] The token cache is being invalidate because project 93da016997ef4f3798af64e5093cd238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:18:56.165277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.166043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.171182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 839/1642] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:18:55 2022] DELETE /identity/v3/projects/93da016997ef4f3798af64e5093cd238 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:56.176346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:56.180233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:56.270592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:56.271499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:56.271952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:56.272371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] PATH_INFO: `/v3/projects/9923d84231fe4263981058d6aa3d0d0b/users/51c6003c0fb1498cbddff5613ffc84a7/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:56.294800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9923d84231fe4263981058d6aa3d0d0b, user_id=51c6003c0fb1498cbddff5613ffc84a7, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:56.297811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:56.344161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 804/1643] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:56 2022] PUT /identity/v3/projects/9923d84231fe4263981058d6aa3d0d0b/users/51c6003c0fb1498cbddff5613ffc84a7/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:56.345541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:56.349808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:56.372411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab67357-a3fa-48fe-bbca-f44b7c6f0eff admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:56.478370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:56.508276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7119-007e-4bb2-8640-8f38d09459cf admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:56.521223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:56.526540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:56.527136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:56.527718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:56.528737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:56.530935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:56.540017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:56.541886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:56.543659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:56.544146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] PATH_INFO: `/v3/users/152b364b885446f0af724255621def53` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:56.548739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=152b364b885446f0af724255621def53)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:56.552665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:56.560843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 805/1644] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:18:56.561808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:18:56.586883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc67a1-c74c-4ac4-89d7-ebbc7a9fa84f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:18:56.606772 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:18:56.615412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01e726-1563-4264-a731-12c38e0dd8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:18:56.615412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e726-1563-4264-a731-12c38e0dd8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:56.615412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e726-1563-4264-a731-12c38e0dd8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:56.615412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e726-1563-4264-a731-12c38e0dd8c0 admin admin] PATH_INFO: `/v3/projects/9923d84231fe4263981058d6aa3d0d0b/users/51c6003c0fb1498cbddff5613ffc84a7/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:56.628866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01e726-1563-4264-a731-12c38e0dd8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9923d84231fe4263981058d6aa3d0d0b, user_id=51c6003c0fb1498cbddff5613ffc84a7, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:18:56.631756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01e726-1563-4264-a731-12c38e0dd8c0 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:18:56.652663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 806/1645] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:18:56 2022] PUT /identity/v3/projects/9923d84231fe4263981058d6aa3d0d0b/users/51c6003c0fb1498cbddff5613ffc84a7/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:18:56.727361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152b364b885446f0af724255621def53'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.755361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152b364b885446f0af724255621def53'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.764587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152b364b885446f0af724255621def53'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.765136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152b364b885446f0af724255621def53'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.765799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152b364b885446f0af724255621def53'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:18:56.807135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 840/1646] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:18:56 2022] DELETE /identity/v3/users/152b364b885446f0af724255621def53 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:18:59.699500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2d4d-d733-4d92-999c-032e42545f80 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:18:59.700967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2d4d-d733-4d92-999c-032e42545f80 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:18:59.701446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2d4d-d733-4d92-999c-032e42545f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:18:59.741025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9acb2d4d-d733-4d92-999c-032e42545f80 None None] MFA Rules not processed for user `51c6003c0fb1498cbddff5613ffc84a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:18:59.742481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9acb2d4d-d733-4d92-999c-032e42545f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:18:59.846209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 807/1647] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:18:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:00.241265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:00.245457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:00.453318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb6104-5e49-45ba-ba74-8f003763bfbe admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:00.469715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:00.472409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:00.474099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:00.474543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] PATH_INFO: `/v3/projects/38ab3c8bad3e4e53b066a9cd25037a7b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:00.480243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ab3c8bad3e4e53b066a9cd25037a7b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:00.485933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:00.612436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ab3c8bad3e4e53b066a9cd25037a7b', 'request_id': 'req-c360eb42-091c-4cee-bd0f-35d3bb84c99d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:00.646608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ab3c8bad3e4e53b066a9cd25037a7b', 'request_id': 'req-c360eb42-091c-4cee-bd0f-35d3bb84c99d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:00.669591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ab3c8bad3e4e53b066a9cd25037a7b', 'request_id': 'req-c360eb42-091c-4cee-bd0f-35d3bb84c99d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:00.703113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] The token cache is being invalidate because project 38ab3c8bad3e4e53b066a9cd25037a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:00.703438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:00.704891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c360eb42-091c-4cee-bd0f-35d3bb84c99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:00.714641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 841/1648] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:00 2022] DELETE /identity/v3/projects/38ab3c8bad3e4e53b066a9cd25037a7b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:00.716586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9da8a-5d52-479b-bce6-d4f751516a92 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:00.717109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9da8a-5d52-479b-bce6-d4f751516a92 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:00.717333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9da8a-5d52-479b-bce6-d4f751516a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:00.765547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-b4f9da8a-5d52-479b-bce6-d4f751516a92 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:00.767161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b4f9da8a-5d52-479b-bce6-d4f751516a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:00.949985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 808/1649] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:00.957575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d88a4-5d34-4438-9ed6-30803c196bc7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:00.957951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d88a4-5d34-4438-9ed6-30803c196bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:00.958310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d88a4-5d34-4438-9ed6-30803c196bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:00.990686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-419d88a4-5d34-4438-9ed6-30803c196bc7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:00.992035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-419d88a4-5d34-4438-9ed6-30803c196bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:01.017494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 842/1650] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:01.018879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9da8a-5d52-479b-bce6-d4f751516a92 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:01.049624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9da8a-5d52-479b-bce6-d4f751516a92 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:01.066455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:01.073031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.073428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.073787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.075819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:01.076221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:01.079181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:01.103286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 809/1651] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:01.103950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d88a4-5d34-4438-9ed6-30803c196bc7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:01.124400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d88a4-5d34-4438-9ed6-30803c196bc7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:01.132273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ccb54-04bb-4701-bf3e-ed2c09f8d118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:01.133757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ccb54-04bb-4701-bf3e-ed2c09f8d118 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.133757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ccb54-04bb-4701-bf3e-ed2c09f8d118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.133757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ccb54-04bb-4701-bf3e-ed2c09f8d118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.134790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ccb54-04bb-4701-bf3e-ed2c09f8d118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:01.136908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ccb54-04bb-4701-bf3e-ed2c09f8d118 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:01.182273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:01.200951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 843/1652] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:01 2022] POST /identity/v3/projects => generated 355 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:01.202183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:01.216322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cfd47-ec34-445b-8bfe-70b3a83bea21 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:01.250089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7596d7-b24b-4692-8dd6-78d8ac488cd4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.250600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7596d7-b24b-4692-8dd6-78d8ac488cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.250699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7596d7-b24b-4692-8dd6-78d8ac488cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.334569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5e7596d7-b24b-4692-8dd6-78d8ac488cd4 None None] MFA Rules not processed for user `6ea7013dd9424b9db1798e81521ee903`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:01.342060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5e7596d7-b24b-4692-8dd6-78d8ac488cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:01.373455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:01.409084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:01.409084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.409084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.409084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.410056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:01.410056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:01.456959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 810/1653] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:01 2022] POST /identity/v3/users => generated 278 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:01.458220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:01.476486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fb74c-6cf1-4997-be22-336ae328951c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:01.491452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:01.493407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.493809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.495961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.497399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:01.505368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:01.538167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 811/1654] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:01.539315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:01.559317 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e565b59-9584-42fe-b9dd-6c8b6687ee2e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:01.574022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28470a-765f-4927-8094-f1a3392ea50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:01.575822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28470a-765f-4927-8094-f1a3392ea50f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.579283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28470a-765f-4927-8094-f1a3392ea50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.579686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28470a-765f-4927-8094-f1a3392ea50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.581787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28470a-765f-4927-8094-f1a3392ea50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:01.590274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28470a-765f-4927-8094-f1a3392ea50f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:01.601431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 844/1655] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:01.625676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 812/1656] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:01.627089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7596d7-b24b-4692-8dd6-78d8ac488cd4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:01.646619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fea49-6486-4b98-b281-4403f83999dd None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.646981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fea49-6486-4b98-b281-4403f83999dd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.647509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fea49-6486-4b98-b281-4403f83999dd None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.649464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 813/1657] 10.222.0.52 () {54 vars in 961 bytes} [Fri Jun 24 09:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:01.655442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7596d7-b24b-4692-8dd6-78d8ac488cd4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:01.668831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab92b9-c1c8-4a18-bc6e-fb134914fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:01.670027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab92b9-c1c8-4a18-bc6e-fb134914fb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.670655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab92b9-c1c8-4a18-bc6e-fb134914fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.670944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab92b9-c1c8-4a18-bc6e-fb134914fb8a admin admin] PATH_INFO: `/v3/projects/46f0ce65859947e9b3b1cbf4362d57da/users/1cd5b7f9108b42e4a637252ec5553926/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.677099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb4c04-1699-4895-a7f9-651cfff93340 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.677680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb4c04-1699-4895-a7f9-651cfff93340 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.678089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb4c04-1699-4895-a7f9-651cfff93340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.743242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a6eb4c04-1699-4895-a7f9-651cfff93340 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:01.747130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a6eb4c04-1699-4895-a7f9-651cfff93340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:01.749073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab92b9-c1c8-4a18-bc6e-fb134914fb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f0ce65859947e9b3b1cbf4362d57da, user_id=1cd5b7f9108b42e4a637252ec5553926, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:01.755095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab92b9-c1c8-4a18-bc6e-fb134914fb8a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:01.793311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 845/1658] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:01 2022] PUT /identity/v3/projects/46f0ce65859947e9b3b1cbf4362d57da/users/1cd5b7f9108b42e4a637252ec5553926/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:01.809089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced404f-75b4-42a0-8057-9fe7f4452308 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:01.809089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced404f-75b4-42a0-8057-9fe7f4452308 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:01.809378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced404f-75b4-42a0-8057-9fe7f4452308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:01.881351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8ced404f-75b4-42a0-8057-9fe7f4452308 None None] MFA Rules not processed for user `1cd5b7f9108b42e4a637252ec5553926`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:01.883438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8ced404f-75b4-42a0-8057-9fe7f4452308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:01.992316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 814/1659] 10.222.0.52 () {58 vars in 1058 bytes} [Fri Jun 24 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:01.993425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb4c04-1699-4895-a7f9-651cfff93340 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:02.053775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb4c04-1699-4895-a7f9-651cfff93340 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:02.131254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 846/1660] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:02.132531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced404f-75b4-42a0-8057-9fe7f4452308 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:02.232070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:02.233242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:02.233242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:02.233451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:02.279610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:02.302948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced404f-75b4-42a0-8057-9fe7f4452308 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:02.312672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:02.313369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:02.313657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:02.313935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:02.318418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:02.320547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:02.327476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:02.358884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:02.365323 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:02.367733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 847/1661] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:02 2022] POST /identity/v3/projects => generated 355 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:02.374694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:02.390526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 815/1662] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:02.463159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8b374-230f-49a2-9373-2522ada19159 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:02.475721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:02.480164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:02.480793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:02.481386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:02.482256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:02.484506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:02.528133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 848/1663] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:02 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:02.532403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:02.553170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf26673-307a-42ed-a8a2-0edc7cddf9ae service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:02.568354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:02.569165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:02.569520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:02.569863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:02.570711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:02.572399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:02.616857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 816/1664] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:02.619316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:02.637999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3242ce-7136-4284-a8f6-6cae7c90b020 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:02.645575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:02.647054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:02.647793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:02.648265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] PATH_INFO: `/v3/projects/cc2cbb23e13849f381574dfecb9aee38/users/4e97a49cef234e9d9fe0d0fa914dfa3c/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:02.670227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc2cbb23e13849f381574dfecb9aee38, user_id=4e97a49cef234e9d9fe0d0fa914dfa3c, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:02.674174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:02.701062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 849/1665] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:02 2022] PUT /identity/v3/projects/cc2cbb23e13849f381574dfecb9aee38/users/4e97a49cef234e9d9fe0d0fa914dfa3c/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:02.702195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:02.720899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44917f-3714-4315-b192-e4a32163278a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:02.864711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49097d-1f20-462a-882f-98c958042810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:02.865421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49097d-1f20-462a-882f-98c958042810 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:02.865808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49097d-1f20-462a-882f-98c958042810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:02.866207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49097d-1f20-462a-882f-98c958042810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:02.868125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49097d-1f20-462a-882f-98c958042810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:02.869960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49097d-1f20-462a-882f-98c958042810 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:02.905191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 817/1666] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:02.910192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:02.924642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a663d11-2794-4193-be48-4bca32c6e842 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:02.950468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:02.952902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:02.953218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:02.954570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] PATH_INFO: `/v3/projects/cc2cbb23e13849f381574dfecb9aee38/users/4e97a49cef234e9d9fe0d0fa914dfa3c/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:02.959629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc2cbb23e13849f381574dfecb9aee38, user_id=4e97a49cef234e9d9fe0d0fa914dfa3c, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:02.968149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:03.015290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0cd9-d394-4deb-8db6-ad280ca502bc None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:03.015583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0cd9-d394-4deb-8db6-ad280ca502bc None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:03.015869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0cd9-d394-4deb-8db6-ad280ca502bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:03.043367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 850/1667] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:02 2022] PUT /identity/v3/projects/cc2cbb23e13849f381574dfecb9aee38/users/4e97a49cef234e9d9fe0d0fa914dfa3c/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:03.113444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-996c0cd9-d394-4deb-8db6-ad280ca502bc None None] MFA Rules not processed for user `4e97a49cef234e9d9fe0d0fa914dfa3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:03.115958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-996c0cd9-d394-4deb-8db6-ad280ca502bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:03.267915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 818/1668] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:03 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:03.280312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:03.282074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:03.417079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758ac5c-fbf6-4b27-bcb0-ac78f5c1eb40 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:03.427994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:03.428698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:03.429285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:03.429566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:03.453378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:03.455089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:03.529215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 851/1669] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:03 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:03.680367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775340fc-4249-4d9f-9b64-99f2509fe83d None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:03.681009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775340fc-4249-4d9f-9b64-99f2509fe83d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:03.681161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775340fc-4249-4d9f-9b64-99f2509fe83d None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:03.683859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 819/1670] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:19:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:03.705819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:03.718204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26be50d-0fd6-4790-a17a-3a4e0213282b service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:03.726601 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38340da-53b8-47da-964c-a1abfd2dde91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:03.727703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38340da-53b8-47da-964c-a1abfd2dde91 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:03.728254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38340da-53b8-47da-964c-a1abfd2dde91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:03.728819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38340da-53b8-47da-964c-a1abfd2dde91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:03.747947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acacdc-aa70-4dea-83c3-d731115d0c2a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:03.749055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acacdc-aa70-4dea-83c3-d731115d0c2a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:03.750004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acacdc-aa70-4dea-83c3-d731115d0c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:03.788156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:03.793578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f8acacdc-aa70-4dea-83c3-d731115d0c2a None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:03.802740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f8acacdc-aa70-4dea-83c3-d731115d0c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:03.884481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38340da-53b8-47da-964c-a1abfd2dde91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:03.885787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38340da-53b8-47da-964c-a1abfd2dde91 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:03.906991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 852/1671] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:03 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:03.987025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:04.069735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 820/1672] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:19:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:04.154929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91371c6-b6aa-4c9c-a294-11075580cbaf None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:04.155208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91371c6-b6aa-4c9c-a294-11075580cbaf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:04.155473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91371c6-b6aa-4c9c-a294-11075580cbaf None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:04.156995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 853/1673] 10.222.0.52 () {54 vars in 960 bytes} [Fri Jun 24 09:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:04.168255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452eb180-ff04-492a-b1c1-b69808ef6bd4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:04.168527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452eb180-ff04-492a-b1c1-b69808ef6bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:04.168927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452eb180-ff04-492a-b1c1-b69808ef6bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:04.219927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-452eb180-ff04-492a-b1c1-b69808ef6bd4 None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:04.221192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-452eb180-ff04-492a-b1c1-b69808ef6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:04.257808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 821/1674] 10.222.0.52 () {58 vars in 1057 bytes} [Fri Jun 24 09:19:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:04.266067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91371c6-b6aa-4c9c-a294-11075580cbaf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:04.298538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91371c6-b6aa-4c9c-a294-11075580cbaf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:04.308461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702ca55-4301-456f-8157-bf2850d8555d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:04.309503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702ca55-4301-456f-8157-bf2850d8555d service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:04.309795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702ca55-4301-456f-8157-bf2850d8555d service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:04.310155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702ca55-4301-456f-8157-bf2850d8555d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:04.323525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702ca55-4301-456f-8157-bf2850d8555d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:04.325820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702ca55-4301-456f-8157-bf2850d8555d service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:04.342484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 854/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:06.393476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9d15-53d6-4d45-b414-ae5a7c2b0ffe None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:06.394047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9d15-53d6-4d45-b414-ae5a7c2b0ffe None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:06.394047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9d15-53d6-4d45-b414-ae5a7c2b0ffe None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:06.396832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 822/1676] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:06.428253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b73c4-fddc-438a-9c34-dc749ab4c80f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:06.429421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b73c4-fddc-438a-9c34-dc749ab4c80f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:06.429871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b73c4-fddc-438a-9c34-dc749ab4c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:06.469546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-043b73c4-fddc-438a-9c34-dc749ab4c80f None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:06.471106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-043b73c4-fddc-438a-9c34-dc749ab4c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:06.482006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 855/1677] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:19:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:06.523523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c9d15-53d6-4d45-b414-ae5a7c2b0ffe None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:06.525455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-495c9d15-53d6-4d45-b414-ae5a7c2b0ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:06.615562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b73c4-fddc-438a-9c34-dc749ab4c80f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:06.628904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b73c4-fddc-438a-9c34-dc749ab4c80f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:06.660257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01201cba-0213-46ba-98b0-f2778b38bf26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:06.661112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01201cba-0213-46ba-98b0-f2778b38bf26 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:06.661400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01201cba-0213-46ba-98b0-f2778b38bf26 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:06.662809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01201cba-0213-46ba-98b0-f2778b38bf26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:06.699149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01201cba-0213-46ba-98b0-f2778b38bf26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:06.704252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01201cba-0213-46ba-98b0-f2778b38bf26 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:06.743529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 856/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:06.754568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c9d15-53d6-4d45-b414-ae5a7c2b0ffe None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:06.771493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:06.772412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:06.772817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:06.775465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] PATH_INFO: `/v3/users/291e03c93abe444a88a3103573a082d0` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:06.778504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291e03c93abe444a88a3103573a082d0)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:06.789714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:06.957807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291e03c93abe444a88a3103573a082d0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:06.982785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291e03c93abe444a88a3103573a082d0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.005106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291e03c93abe444a88a3103573a082d0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.008668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291e03c93abe444a88a3103573a082d0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.009733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291e03c93abe444a88a3103573a082d0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.043412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b63e8-499d-4ca1-ac12-01b99119fa80 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:07.043801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b63e8-499d-4ca1-ac12-01b99119fa80 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:07.044152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b63e8-499d-4ca1-ac12-01b99119fa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:07.119603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 823/1679] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:06 2022] DELETE /identity/v3/users/291e03c93abe444a88a3103573a082d0 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:07.126499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e94b63e8-499d-4ca1-ac12-01b99119fa80 None None] MFA Rules not processed for user `f8333f6a9dab4618bc0e6b2060f15250`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:07.129662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e94b63e8-499d-4ca1-ac12-01b99119fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:07.198519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:07.199996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:07.274953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 857/1680] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:07.340730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24b19c-9957-4774-9d3e-c35f811bce89 None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:07.341389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24b19c-9957-4774-9d3e-c35f811bce89 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:07.341389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24b19c-9957-4774-9d3e-c35f811bce89 None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:07.343246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 858/1681] 10.222.0.52 () {54 vars in 961 bytes} [Fri Jun 24 09:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:07.371345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055337ab-b887-4c67-b325-1dd5e43654d9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:07.394845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:07.395763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:07.396047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:07.396319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] PATH_INFO: `/v3/users/6063674fe3064e41b89e91533c3e0258` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:07.399071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6063674fe3064e41b89e91533c3e0258)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:07.402657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1ca77-c5a0-4aa9-9dd2-b47d1d5af840 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:07.403154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1ca77-c5a0-4aa9-9dd2-b47d1d5af840 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:07.403502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1ca77-c5a0-4aa9-9dd2-b47d1d5af840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:07.405679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:07.412397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:07.442272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7dd1ca77-c5a0-4aa9-9dd2-b47d1d5af840 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:07.449147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7dd1ca77-c5a0-4aa9-9dd2-b47d1d5af840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:07.531718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6063674fe3064e41b89e91533c3e0258'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.548507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6063674fe3064e41b89e91533c3e0258'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.571650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6063674fe3064e41b89e91533c3e0258'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.575141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6063674fe3064e41b89e91533c3e0258'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.577047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42595163-e3bd-43ef-bc5b-14cd344928e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6063674fe3064e41b89e91533c3e0258'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:07.628915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 824/1682] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:07 2022] DELETE /identity/v3/users/6063674fe3064e41b89e91533c3e0258 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:07.650093 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:07.652285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:07.653861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:07.698706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 859/1683] 10.222.0.52 () {58 vars in 1058 bytes} [Fri Jun 24 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:07.700238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1ca77-c5a0-4aa9-9dd2-b47d1d5af840 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:07.738645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1ca77-c5a0-4aa9-9dd2-b47d1d5af840 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:07.754255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:07.759480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:07.797869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:07.951457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430efaf7-e62f-44e9-ab09-a92df4419d66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:07.952264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430efaf7-e62f-44e9-ab09-a92df4419d66 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:07.955449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430efaf7-e62f-44e9-ab09-a92df4419d66 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:07.956022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430efaf7-e62f-44e9-ab09-a92df4419d66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:07.962027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-430efaf7-e62f-44e9-ab09-a92df4419d66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.015839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 825/1684] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:08.107922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430efaf7-e62f-44e9-ab09-a92df4419d66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:08.111867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430efaf7-e62f-44e9-ab09-a92df4419d66 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:08.140660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 860/1685] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:08.142970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:08.145509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.173595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e35bdb-f884-408f-aa4a-5b6c6ca4f46c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:08.173971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e35bdb-f884-408f-aa4a-5b6c6ca4f46c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:08.174332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e35bdb-f884-408f-aa4a-5b6c6ca4f46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:08.227953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-25e35bdb-f884-408f-aa4a-5b6c6ca4f46c None None] MFA Rules not processed for user `592836cc56ed4334bb46890a74460307`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:08.230846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-25e35bdb-f884-408f-aa4a-5b6c6ca4f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.349148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa675e-4ffd-4a9e-9c17-fb8b69376e79 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:08.354336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:08.355313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:08.355822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:08.356418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:08.370253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:08.372262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:08.403390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 861/1686] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:08.406417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:08.408425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:08.408425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:08.412346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 826/1687] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:08.436711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] MFA Rules not processed for user `592836cc56ed4334bb46890a74460307`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:08.437931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.461874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 862/1688] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:08.469772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:08.472540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.516418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106174f-42ef-4dba-87e4-707612fce063 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:08.525876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c231a3-d59f-440b-9f5d-4d52c3b79139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:08.527391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c231a3-d59f-440b-9f5d-4d52c3b79139 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:08.527889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c231a3-d59f-440b-9f5d-4d52c3b79139 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:08.529115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c231a3-d59f-440b-9f5d-4d52c3b79139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:08.548727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c231a3-d59f-440b-9f5d-4d52c3b79139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:08.552492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c231a3-d59f-440b-9f5d-4d52c3b79139 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:08.581739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 827/1689] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:08.662033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:08.663769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.696266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b433c-109e-4de3-84e5-5da2c4b4d237 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:08.719345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f859c-0611-4c78-b6f9-64d5ca5b64a0 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:08.719856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f859c-0611-4c78-b6f9-64d5ca5b64a0 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:08.720298 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f859c-0611-4c78-b6f9-64d5ca5b64a0 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:08.722449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 828/1690] 10.222.0.52 () {54 vars in 912 bytes} [Fri Jun 24 09:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:19:08.723402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:08.724453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:08.724888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:08.726454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] PATH_INFO: `/v3/projects/bb248b98ec2e4011b059436cef046c14` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:08.730616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb248b98ec2e4011b059436cef046c14)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:08.732820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:08.742720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:08.742980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:08.743256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:08.783432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:08.785347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.824208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 829/1691] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:19:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:08.841177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:08.843072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:08.853582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:08.881317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:08.927116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb248b98ec2e4011b059436cef046c14', 'request_id': 'req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:08.951078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb248b98ec2e4011b059436cef046c14', 'request_id': 'req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:08.961765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb248b98ec2e4011b059436cef046c14', 'request_id': 'req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:08.986547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480021-1302-4157-a8fe-40f4dbb7c2f9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:08.994360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] The token cache is being invalidate because project bb248b98ec2e4011b059436cef046c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:08.994660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:08.995356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:08.997374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 863/1692] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:08 2022] DELETE /identity/v3/projects/bb248b98ec2e4011b059436cef046c14 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:09.002684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:09.004283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:09.113123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:09.113934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:09.114268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:09.114847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:09.123332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:09.232775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3f8b8-1e87-4ab6-b4c9-9d77c9a152c6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:09.241156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:09.242537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:09.242825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:09.243291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] PATH_INFO: `/v3/users/51c6003c0fb1498cbddff5613ffc84a7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:09.244565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c6003c0fb1498cbddff5613ffc84a7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:09.247155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:09.319631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:09.321138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:09.352971 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 830/1693] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:09.402002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c6003c0fb1498cbddff5613ffc84a7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:09.441524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c6003c0fb1498cbddff5613ffc84a7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:09.457511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c6003c0fb1498cbddff5613ffc84a7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:09.460032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c6003c0fb1498cbddff5613ffc84a7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:09.461056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c6003c0fb1498cbddff5613ffc84a7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:09.536520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 864/1694] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:09 2022] DELETE /identity/v3/users/51c6003c0fb1498cbddff5613ffc84a7 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:10.537336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:10.538990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:10.582229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49828b-4341-4a12-9b75-5c990be5b70f service glance] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:10.762081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:10.762975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:10.763254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:10.763685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] PATH_INFO: `/v3/projects/836bfadb581e4e2fb7a5514f8ef6fff9` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:10.770723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=836bfadb581e4e2fb7a5514f8ef6fff9)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:10.771790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: WARNING py.warnings [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:19:10.771790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: warnings.warn(msg) Jun 24 09:19:10.771790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: Jun 24 09:19:10.773041 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:10.776888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:10.905731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836bfadb581e4e2fb7a5514f8ef6fff9', 'request_id': 'req-8474c713-ee27-4b97-8a25-2f807c5de8a1'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:10.910262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:10.914433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:10.936597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf5850-38fb-4c1d-9e3d-cc2be88018c9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:10.942939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836bfadb581e4e2fb7a5514f8ef6fff9', 'request_id': 'req-8474c713-ee27-4b97-8a25-2f807c5de8a1'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:10.974652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836bfadb581e4e2fb7a5514f8ef6fff9', 'request_id': 'req-8474c713-ee27-4b97-8a25-2f807c5de8a1'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.004100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] The token cache is being invalidate because project 836bfadb581e4e2fb7a5514f8ef6fff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:11.004322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.004988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8474c713-ee27-4b97-8a25-2f807c5de8a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.006977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 831/1695] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:10 2022] DELETE /identity/v3/projects/836bfadb581e4e2fb7a5514f8ef6fff9 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:11.008748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746145-193b-4d62-91d4-2aff91028616 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.009041 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746145-193b-4d62-91d4-2aff91028616 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.009464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746145-193b-4d62-91d4-2aff91028616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.043161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-33746145-193b-4d62-91d4-2aff91028616 None None] MFA Rules not processed for user `6ea7013dd9424b9db1798e81521ee903`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:11.044773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-33746145-193b-4d62-91d4-2aff91028616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:11.084922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:11.127980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:11.129380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.130206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.130727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] PATH_INFO: `/v3/projects/9923d84231fe4263981058d6aa3d0d0b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.133240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9923d84231fe4263981058d6aa3d0d0b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:11.137476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:11.277012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 832/1696] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:11.278632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc001a-d59b-4d49-91d6-0b2a2ff85a15 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.278928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc001a-d59b-4d49-91d6-0b2a2ff85a15 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.282479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc001a-d59b-4d49-91d6-0b2a2ff85a15 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.286367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 833/1697] 10.222.0.52 () {54 vars in 912 bytes} [Fri Jun 24 09:19:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:11.287213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc001a-d59b-4d49-91d6-0b2a2ff85a15 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:11.289069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-70bc001a-d59b-4d49-91d6-0b2a2ff85a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:11.335111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc001a-d59b-4d49-91d6-0b2a2ff85a15 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:11.358511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:11.359469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.361813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.365781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] PATH_INFO: `/v3/users/4b026e302a954d8eb9d6c2c3a2857368` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.367541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b026e302a954d8eb9d6c2c3a2857368)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:11.372533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:11.413537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9923d84231fe4263981058d6aa3d0d0b', 'request_id': 'req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.434066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9923d84231fe4263981058d6aa3d0d0b', 'request_id': 'req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.445760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9923d84231fe4263981058d6aa3d0d0b', 'request_id': 'req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.490168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] The token cache is being invalidate because project 9923d84231fe4263981058d6aa3d0d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:11.491130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.494759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-35cfb2cb-3ce5-49ff-97d4-0a898b9114a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.499243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 865/1698] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:10 2022] DELETE /identity/v3/projects/9923d84231fe4263981058d6aa3d0d0b => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:11.504930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0ce27-7f02-4a24-a166-ed67d247567a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.506031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0ce27-7f02-4a24-a166-ed67d247567a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.507467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0ce27-7f02-4a24-a166-ed67d247567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.512814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b026e302a954d8eb9d6c2c3a2857368'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.528756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b026e302a954d8eb9d6c2c3a2857368'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.549519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b026e302a954d8eb9d6c2c3a2857368'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.563155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b026e302a954d8eb9d6c2c3a2857368'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.564808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-fa686d03-2d1f-48b9-8653-788128648ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b026e302a954d8eb9d6c2c3a2857368'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:11.584978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b9f0ce27-7f02-4a24-a166-ed67d247567a None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:11.586499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b9f0ce27-7f02-4a24-a166-ed67d247567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:11.617830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 834/1699] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:11 2022] DELETE /identity/v3/users/4b026e302a954d8eb9d6c2c3a2857368 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:11.619299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e83036-d4e6-4c94-b8fc-7fc7792c9bc6 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.619594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e83036-d4e6-4c94-b8fc-7fc7792c9bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.622138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e83036-d4e6-4c94-b8fc-7fc7792c9bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.689493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:11.719467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-43e83036-d4e6-4c94-b8fc-7fc7792c9bc6 None None] MFA Rules not processed for user `00556257371a41b39bce3ca8ee545238`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:11.723170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-43e83036-d4e6-4c94-b8fc-7fc7792c9bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:11.771214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 866/1700] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:11.772862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3293e-2109-418d-b2b0-8d8ef0b09e2d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.773143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3293e-2109-418d-b2b0-8d8ef0b09e2d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.773426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3293e-2109-418d-b2b0-8d8ef0b09e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.853048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 835/1701] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:11.854637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd1f4-4201-4c4a-a2fb-ffb8e53af239 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.855086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd1f4-4201-4c4a-a2fb-ffb8e53af239 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.855465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd1f4-4201-4c4a-a2fb-ffb8e53af239 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.857242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 836/1702] 10.222.0.52 () {54 vars in 912 bytes} [Fri Jun 24 09:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:11.858397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ad99-c1d5-4ea9-9d6f-e6495ee04417 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:11.858794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ad99-c1d5-4ea9-9d6f-e6495ee04417 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:11.859168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ad99-c1d5-4ea9-9d6f-e6495ee04417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:11.865434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6ca3293e-2109-418d-b2b0-8d8ef0b09e2d None None] MFA Rules not processed for user `b97b26dd3bb5427a8edfe4eb653b945b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:11.867163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6ca3293e-2109-418d-b2b0-8d8ef0b09e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:11.883965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-4126ad99-c1d5-4ea9-9d6f-e6495ee04417 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:11.885885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4126ad99-c1d5-4ea9-9d6f-e6495ee04417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:12.004881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 837/1703] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:12.007175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a87e-bd42-418f-aa41-0a49c0a14716 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.007594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a87e-bd42-418f-aa41-0a49c0a14716 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.008117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a87e-bd42-418f-aa41-0a49c0a14716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.057707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:12.085235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 867/1704] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:12.086783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc43db9-916f-49fd-a81c-97f830939ec7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.087074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc43db9-916f-49fd-a81c-97f830939ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.087368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc43db9-916f-49fd-a81c-97f830939ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.096899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8976a87e-bd42-418f-aa41-0a49c0a14716 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:12.098909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8976a87e-bd42-418f-aa41-0a49c0a14716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:12.125129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7fc43db9-916f-49fd-a81c-97f830939ec7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:12.130885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7fc43db9-916f-49fd-a81c-97f830939ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:12.147580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 868/1705] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:12.152268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.152581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.152853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.191956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] MFA Rules not processed for user `b97b26dd3bb5427a8edfe4eb653b945b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:12.193223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:12.201889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 838/1706] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:12.203503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976a87e-bd42-418f-aa41-0a49c0a14716 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.211488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 869/1707] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:12.217197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.218813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:12.224204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976a87e-bd42-418f-aa41-0a49c0a14716 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.229258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.230164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.230451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.230726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.231592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:12.232274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.234098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.254765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 839/1708] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:12.257173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.274886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a7160-80a3-460a-b6a6-ca750696fa16 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.286435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.288388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.288910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.289646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.292275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.295613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.316898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:12.336622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 840/1709] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:12 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:12.340906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.353806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716a141-6e4f-4c49-9f53-a278312104cf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.465539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.467125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.467618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.468081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.509356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dd99b-b7c9-48fc-ad66-96af67885e21 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.528840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.530321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.530907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.535963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.537503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.540034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.563243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.567249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.578285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 841/1710] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:12 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:12.580210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.584092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:12.591901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 870/1711] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:12.592583 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.605266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51de8b-f253-43de-89f3-9d6c1037636c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.611698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.613458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.613929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.614498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] PATH_INFO: `/v3/projects/0cfd69768cd04e509e2aca6d77b24129` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.616053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cfd69768cd04e509e2aca6d77b24129)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.621529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.623107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56e26-ad29-438b-8faf-d9dc590f369a service glance] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.633089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.633750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.634034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.634310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.635131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.637455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.648857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 871/1712] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:12.652542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.664683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991c2a7-3af1-4f09-bcf1-2dcbfcb88370 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.672650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.672650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.672650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.673150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.674178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.676819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.692241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 872/1713] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:12.697130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:12.714063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3776764-714e-431c-b7e1-d15fa9cf8ea7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:12.723014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22390e-4129-496d-92ad-9dfa2162e230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:12.724101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22390e-4129-496d-92ad-9dfa2162e230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.724457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22390e-4129-496d-92ad-9dfa2162e230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.725042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22390e-4129-496d-92ad-9dfa2162e230 admin admin] PATH_INFO: `/v3/projects/05d6314a250b4e83b8ad1768b4ae3c89/users/dd93915267ab466fb60c1ff14b99c16d/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.753616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22390e-4129-496d-92ad-9dfa2162e230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d6314a250b4e83b8ad1768b4ae3c89, user_id=dd93915267ab466fb60c1ff14b99c16d, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:12.756929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22390e-4129-496d-92ad-9dfa2162e230 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:12.758418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cfd69768cd04e509e2aca6d77b24129', 'request_id': 'req-4113376b-dff3-4771-ab4f-3c5fec704e9f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:12.761448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:12.773202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cfd69768cd04e509e2aca6d77b24129', 'request_id': 'req-4113376b-dff3-4771-ab4f-3c5fec704e9f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:12.786690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 873/1714] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:12 2022] PUT /identity/v3/projects/05d6314a250b4e83b8ad1768b4ae3c89/users/dd93915267ab466fb60c1ff14b99c16d/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:12.787462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cfd69768cd04e509e2aca6d77b24129', 'request_id': 'req-4113376b-dff3-4771-ab4f-3c5fec704e9f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:12.796919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:12.797340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:12.797733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:12.864077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] MFA Rules not processed for user `dd93915267ab466fb60c1ff14b99c16d`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:12.865573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:13.118276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 874/1715] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:13.133668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:13.135748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:13.211650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] The token cache is being invalidate because project 0cfd69768cd04e509e2aca6d77b24129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:13.212075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:13.217103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:13.223683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 842/1716] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:12 2022] DELETE /identity/v3/projects/0cfd69768cd04e509e2aca6d77b24129 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:13.223683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:13.225674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:13.261076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0ccd5-5ceb-43e2-abcd-a97297eff924 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:13.271223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3c618-3edb-4746-86b2-1eaf1021fe28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:13.271897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3c618-3edb-4746-86b2-1eaf1021fe28 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:13.272182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3c618-3edb-4746-86b2-1eaf1021fe28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:13.272462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3c618-3edb-4746-86b2-1eaf1021fe28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:13.278040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d3f3c618-3edb-4746-86b2-1eaf1021fe28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:13.301838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3c618-3edb-4746-86b2-1eaf1021fe28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:13.307791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3c618-3edb-4746-86b2-1eaf1021fe28 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:13.348763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 875/1717] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:13.350781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb5af-7cef-4513-b01f-625d17ed2ed8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:13.351068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb5af-7cef-4513-b01f-625d17ed2ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:13.351347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb5af-7cef-4513-b01f-625d17ed2ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:13.425361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113376b-dff3-4771-ab4f-3c5fec704e9f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:13.440957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ec4cb5af-7cef-4513-b01f-625d17ed2ed8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:13.448794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ec4cb5af-7cef-4513-b01f-625d17ed2ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:13.465026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:13.589723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:13.589723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:13.589723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:13.589723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:13.590578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:13.594456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:13.650373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 876/1718] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:13.658495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f749e7-288a-46a8-8431-9eea2d08a464 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:13.658829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f749e7-288a-46a8-8431-9eea2d08a464 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:13.659110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f749e7-288a-46a8-8431-9eea2d08a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:13.680865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 843/1719] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:13 2022] POST /identity/v3/projects => generated 365 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:13.681958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:13.696280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a6f749e7-288a-46a8-8431-9eea2d08a464 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:13.701407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a6f749e7-288a-46a8-8431-9eea2d08a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:13.821605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:13.837874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efda23-8dac-4043-9270-73e7a870269f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:13.845752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe51120-518c-4fe0-8963-09d8b18f40d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:13.847352 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe51120-518c-4fe0-8963-09d8b18f40d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:13.848181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe51120-518c-4fe0-8963-09d8b18f40d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:13.848181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe51120-518c-4fe0-8963-09d8b18f40d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:13.849223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe51120-518c-4fe0-8963-09d8b18f40d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:13.856138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe51120-518c-4fe0-8963-09d8b18f40d9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:13.929648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 844/1720] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:13 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:13.931591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcfd3-dcdd-41f8-b860-f3eda493eb74 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:13.932927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcfd3-dcdd-41f8-b860-f3eda493eb74 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:13.933443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcfd3-dcdd-41f8-b860-f3eda493eb74 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:13.939140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 845/1721] 10.222.0.52 () {54 vars in 912 bytes} [Fri Jun 24 09:19:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:13.941530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdcfd3-dcdd-41f8-b860-f3eda493eb74 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:13.957562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 877/1722] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:13.961808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92537b9d-e9cf-485b-96a3-f6dfb0280b31 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:13.962223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92537b9d-e9cf-485b-96a3-f6dfb0280b31 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:13.962510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92537b9d-e9cf-485b-96a3-f6dfb0280b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:13.985230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdcfd3-dcdd-41f8-b860-f3eda493eb74 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:13.997330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.012962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.013510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.013895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.020834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.027630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.030804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-92537b9d-e9cf-485b-96a3-f6dfb0280b31 None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:14.042253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-92537b9d-e9cf-485b-96a3-f6dfb0280b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:14.069887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 846/1723] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:14.084603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.113083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cabcee-d944-49f5-b284-d4f58de92c71 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:14.126896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.128471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.128974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.129457 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.130534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:14.131982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.134039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.154170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 847/1724] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:14.155201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.168382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2fd55-b175-4f38-8c13-5dabfc30d491 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:14.183544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.184686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.185128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.185604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] PATH_INFO: `/v3/projects/3962d4b1729d48bf8bac4a4273288601/users/518dab278e0a49538b7d12024de76748/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.249172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3962d4b1729d48bf8bac4a4273288601, user_id=518dab278e0a49538b7d12024de76748, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.266198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.286439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 878/1725] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:14.287873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92537b9d-e9cf-485b-96a3-f6dfb0280b31 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.309281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92537b9d-e9cf-485b-96a3-f6dfb0280b31 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:14.314329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 848/1726] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:14 2022] PUT /identity/v3/projects/3962d4b1729d48bf8bac4a4273288601/users/518dab278e0a49538b7d12024de76748/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:14.315195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.318067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:14.392440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:14.411758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dc39c-0b77-41cf-82f6-29b64f35d11c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:14.417332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.418103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.418471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.418931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.429610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.431992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.484312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 849/1727] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:14.485666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.486709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.489718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.490090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.490692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.492016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.499016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.514794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883d7dc-6c0a-4913-b9f2-5e1e6699b088 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:14.535876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe03256-f7bd-4cb6-9e36-cfb79ef7fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.538339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe03256-f7bd-4cb6-9e36-cfb79ef7fb70 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.539358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe03256-f7bd-4cb6-9e36-cfb79ef7fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.539874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe03256-f7bd-4cb6-9e36-cfb79ef7fb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.541742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe03256-f7bd-4cb6-9e36-cfb79ef7fb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.557234 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe03256-f7bd-4cb6-9e36-cfb79ef7fb70 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.579425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 850/1728] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:14.582804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9078505-6587-408a-ae71-2221e268cb44 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.584471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9078505-6587-408a-ae71-2221e268cb44 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.584787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9078505-6587-408a-ae71-2221e268cb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.594472 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 879/1729] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:14 2022] POST /identity/v3/projects => generated 367 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:14.599138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.626588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a9078505-6587-408a-ae71-2221e268cb44 None None] MFA Rules not processed for user `f8333f6a9dab4618bc0e6b2060f15250`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:14.630515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a9078505-6587-408a-ae71-2221e268cb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:14.739018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1291ee-41a6-467f-b851-1c595014e5c4 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:14.766296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.767612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.774223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.774223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] PATH_INFO: `/v3/projects/3962d4b1729d48bf8bac4a4273288601/users/518dab278e0a49538b7d12024de76748/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.775245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 851/1730] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:14.777432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9078505-6587-408a-ae71-2221e268cb44 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.779108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3962d4b1729d48bf8bac4a4273288601, user_id=518dab278e0a49538b7d12024de76748, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.783186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.798195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:14.801450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9078505-6587-408a-ae71-2221e268cb44 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:14.814557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 880/1731] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:14 2022] PUT /identity/v3/projects/3962d4b1729d48bf8bac4a4273288601/users/518dab278e0a49538b7d12024de76748/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:14.816977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:14.819253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:14.908343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48868a7f-446f-4a0f-91d5-438b9a03360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:14.909504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48868a7f-446f-4a0f-91d5-438b9a03360e admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.910032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48868a7f-446f-4a0f-91d5-438b9a03360e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.910441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48868a7f-446f-4a0f-91d5-438b9a03360e admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.911287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48868a7f-446f-4a0f-91d5-438b9a03360e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:14.914295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48868a7f-446f-4a0f-91d5-438b9a03360e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:14.947509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 852/1732] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:14 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:14.949626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:14.950441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:14.950890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:14.990575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] MFA Rules not processed for user `518dab278e0a49538b7d12024de76748`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:14.997889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.012687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad53dee-baae-4445-a705-944c1bd162c8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.039928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.040956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.041440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.041933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.045015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.093928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:15.140696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 853/1733] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:15.140696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:15.141699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.180394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.182396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.199802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 881/1734] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:15.200718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:15.214126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9228a-e200-4286-8b8c-b60ffbc8c955 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.224085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.224851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.225141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.225420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.226401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.228508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.275253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 882/1735] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:15.276082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:15.289374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.349035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d7df7-8cea-495b-b1cc-b9fa812e23de admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.366701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.367454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.367809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.368159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.375775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.400940 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c31e45-9151-4bcf-bcbf-bc9318c72b6c None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.420032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.421369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.422156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.423572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.432206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.469701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.474043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.525741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.528258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.533500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 854/1736] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:15.537486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:15.541382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.565058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 883/1737] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:15.566987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:15.569619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:15.618789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00556bfb-6c91-4879-ae63-c4dcab89ed2d service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.628399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:15.638254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.643498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.647647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.650694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.676262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.677948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.730276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 855/1738] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:15.732038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:15.736869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10de3-80cd-45fb-8658-f802112afb97 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.748849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a45a26-481d-459e-91a1-e46ba3202c85 service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.751926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749456a1-4ebe-4f37-a948-650d3b66814c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.752771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749456a1-4ebe-4f37-a948-650d3b66814c service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.753447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749456a1-4ebe-4f37-a948-650d3b66814c service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.753447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749456a1-4ebe-4f37-a948-650d3b66814c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.756460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.757318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.757939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.759938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.760910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.762884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.805978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749456a1-4ebe-4f37-a948-650d3b66814c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.807457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749456a1-4ebe-4f37-a948-650d3b66814c service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.818531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 856/1739] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:15.825489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:15.862995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 884/1740] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:15.866553 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd3a9-c96b-4302-8e76-7c22a2491ade admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:15.880876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:15.883084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.883315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.883595 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] PATH_INFO: `/v3/projects/c84371bb67474ccabf04d10c9fc6f2ea/users/32cfe1d8aeba44b5bb500c3234220aac/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:15.926175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84371bb67474ccabf04d10c9fc6f2ea, user_id=32cfe1d8aeba44b5bb500c3234220aac, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:15.930910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:15.961096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 857/1741] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:15 2022] PUT /identity/v3/projects/c84371bb67474ccabf04d10c9fc6f2ea/users/32cfe1d8aeba44b5bb500c3234220aac/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:15.972346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f188c6-97b4-4165-9380-dfdbf4b21bf8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:15.972732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f188c6-97b4-4165-9380-dfdbf4b21bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:15.973068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f188c6-97b4-4165-9380-dfdbf4b21bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:16.050120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-83f188c6-97b4-4165-9380-dfdbf4b21bf8 None None] MFA Rules not processed for user `32cfe1d8aeba44b5bb500c3234220aac`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:16.050120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-83f188c6-97b4-4165-9380-dfdbf4b21bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:16.078404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:16.320313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be1e72-b9a4-4c7e-80ae-c1eaa5cefb17 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:16.339175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:16.340248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:16.340248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:16.340437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:16.348878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:16.387096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 885/1742] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:16.396084 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f188c6-97b4-4165-9380-dfdbf4b21bf8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:16.623043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:16.653357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:16.666505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:16.693410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 858/1743] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:16.766756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f188c6-97b4-4165-9380-dfdbf4b21bf8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:16.776692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:16.777946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:16.778527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:16.778943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:16.780001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:16.782228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:16.897140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 886/1744] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:16 2022] POST /identity/v3/projects => generated 365 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:16.899494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:16.933596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3fd1e-fb05-42b5-b393-9fd9192411be service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:17.224815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:17.228608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:17.229278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:17.229584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:17.231646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:17.236169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:17.299137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 859/1745] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:16 2022] POST /identity/v3/users => generated 289 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:17.301561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:17.326959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57837e-e38c-47c7-b7a5-d889ec22e6f7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:17.343085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:17.348353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:17.349229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:17.350656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:17.352781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:17.360235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:17.403090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 887/1746] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:17.406280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:17.422262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752f06d-b59a-48c1-9e35-69da0288c76b admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:17.440513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:17.446794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:17.448070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:17.448511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] PATH_INFO: `/v3/projects/99ee61e24eb6418bbc51400af6e15066/users/4a526b45bfc4477ea3f113d7f04ecd3d/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:17.506356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ee61e24eb6418bbc51400af6e15066, user_id=4a526b45bfc4477ea3f113d7f04ecd3d, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:17.511211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:17.574546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 860/1747] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:17 2022] PUT /identity/v3/projects/99ee61e24eb6418bbc51400af6e15066/users/4a526b45bfc4477ea3f113d7f04ecd3d/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:17.577532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:17.615335 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace917e-fdb7-4817-a3b0-2f915b4b99c2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:17.808770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:17.967709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e6a85-a6e9-40ed-9e59-c98fb0c83c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:17.983012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e6a85-a6e9-40ed-9e59-c98fb0c83c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:17.983545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e6a85-a6e9-40ed-9e59-c98fb0c83c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:17.984953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e6a85-a6e9-40ed-9e59-c98fb0c83c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:17.987843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e6a85-a6e9-40ed-9e59-c98fb0c83c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:17.996784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e6a85-a6e9-40ed-9e59-c98fb0c83c74 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:18.040286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:18.047122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 888/1748] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:18.069274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ba43f-3535-4260-bf69-d8ed55ca2765 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:18.090403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:18.094023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:18.097859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:18.098875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] PATH_INFO: `/v3/projects/99ee61e24eb6418bbc51400af6e15066/users/4a526b45bfc4477ea3f113d7f04ecd3d/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:18.109522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ee61e24eb6418bbc51400af6e15066, user_id=4a526b45bfc4477ea3f113d7f04ecd3d, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:18.120048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:18.212749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 861/1749] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:18 2022] PUT /identity/v3/projects/99ee61e24eb6418bbc51400af6e15066/users/4a526b45bfc4477ea3f113d7f04ecd3d/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:18.224975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330304-ecdd-43d8-a540-989a1f98d038 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:18.225409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330304-ecdd-43d8-a540-989a1f98d038 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:18.225802 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330304-ecdd-43d8-a540-989a1f98d038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:18.301708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-88330304-ecdd-43d8-a540-989a1f98d038 None None] MFA Rules not processed for user `4a526b45bfc4477ea3f113d7f04ecd3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:18.303118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-88330304-ecdd-43d8-a540-989a1f98d038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:18.515283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 889/1750] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:18.681462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:18.683692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:18.857732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdbf42-09d3-4b2e-a0c7-fafb492b98ac admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:18.865142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c754ed-dbd1-4d47-b256-f56c0c63ca83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:18.865929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c754ed-dbd1-4d47-b256-f56c0c63ca83 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:18.866210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c754ed-dbd1-4d47-b256-f56c0c63ca83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:18.866485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c754ed-dbd1-4d47-b256-f56c0c63ca83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:18.898773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c754ed-dbd1-4d47-b256-f56c0c63ca83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:18.901647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c754ed-dbd1-4d47-b256-f56c0c63ca83 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:18.917636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 862/1751] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:19.091591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330304-ecdd-43d8-a540-989a1f98d038 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:19.130647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330304-ecdd-43d8-a540-989a1f98d038 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:19.155311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df84f-a613-4914-9657-5163a146e331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:19.156449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df84f-a613-4914-9657-5163a146e331 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:19.157012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df84f-a613-4914-9657-5163a146e331 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:19.157504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df84f-a613-4914-9657-5163a146e331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:19.239675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:19.240716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:19.241223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:19.287415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] MFA Rules not processed for user `b97b26dd3bb5427a8edfe4eb653b945b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:19.290856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:19.407040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df84f-a613-4914-9657-5163a146e331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:19.410457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df84f-a613-4914-9657-5163a146e331 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:19.462121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 890/1752] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:19.490409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:19.550508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 863/1753] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:20.727642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be15d6-dbd0-4846-b036-f7b6aba55038 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:20.728586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be15d6-dbd0-4846-b036-f7b6aba55038 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:20.729131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be15d6-dbd0-4846-b036-f7b6aba55038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:20.770540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a1be15d6-dbd0-4846-b036-f7b6aba55038 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:20.773745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a1be15d6-dbd0-4846-b036-f7b6aba55038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:20.965411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:20.989854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 891/1754] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:19:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:21.319854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:21.321544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:21.626769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae583a1-4cd2-4ac9-9628-6700320002ca None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:21.647020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:21.654363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:21.654702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:21.654989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:21.683441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:21.686405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:21.728715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 864/1755] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:19:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:19:24.476122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:24.476831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:24.477078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:24.534876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] MFA Rules not processed for user `00556257371a41b39bce3ca8ee545238`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:24.541115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:24.862919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 892/1756] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:28.516375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:28.664089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:28.664754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7d50b-07ab-4f18-b284-34ffc994e3e5 service swift] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:28.667742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:28.670248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:28.676832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:28.677394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:28.677823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] PATH_INFO: `/v3/users/32cfe1d8aeba44b5bb500c3234220aac` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:28.679693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32cfe1d8aeba44b5bb500c3234220aac)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:28.681800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:28.692357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd17c8-dd93-4790-99a0-9a2c0bb86de8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:28.703368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:28.704469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:28.706057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:28.706459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] PATH_INFO: `/v3/users/1cd5b7f9108b42e4a637252ec5553926` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:28.708212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd5b7f9108b42e4a637252ec5553926)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:28.710622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:28.786384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32cfe1d8aeba44b5bb500c3234220aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.803033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32cfe1d8aeba44b5bb500c3234220aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.816228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd5b7f9108b42e4a637252ec5553926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.821526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32cfe1d8aeba44b5bb500c3234220aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.824540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32cfe1d8aeba44b5bb500c3234220aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.825697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a8191898-0bd8-4582-9817-b610e8f9d6a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32cfe1d8aeba44b5bb500c3234220aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.831435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd5b7f9108b42e4a637252ec5553926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.851846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd5b7f9108b42e4a637252ec5553926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.853692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd5b7f9108b42e4a637252ec5553926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.855062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5b39be-0644-4597-bd02-622ae391e389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd5b7f9108b42e4a637252ec5553926'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:28.867861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:28.899878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 865/1758] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:28 2022] DELETE /identity/v3/users/32cfe1d8aeba44b5bb500c3234220aac => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:28.907518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 893/1758] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:28 2022] DELETE /identity/v3/users/1cd5b7f9108b42e4a637252ec5553926 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:28.910025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:28.910776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:28.911478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:28.919577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba535bc-4a85-4d98-9b23-af4143287c07 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:28.919966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba535bc-4a85-4d98-9b23-af4143287c07 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:28.920595 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba535bc-4a85-4d98-9b23-af4143287c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:28.943118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:28.945039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:28.959421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-eba535bc-4a85-4d98-9b23-af4143287c07 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:28.961588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-eba535bc-4a85-4d98-9b23-af4143287c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:29.081776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 894/1759] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:29.154038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:29.154038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:29.183522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:29.255938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 866/1760] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:29.257893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7b2e-ab0b-4193-b587-47dd425f4a7a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:29.259719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7b2e-ab0b-4193-b587-47dd425f4a7a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:29.264791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7b2e-ab0b-4193-b587-47dd425f4a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:29.339343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a8dc7b2e-ab0b-4193-b587-47dd425f4a7a None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:29.344346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a8dc7b2e-ab0b-4193-b587-47dd425f4a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:29.369888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bdc71-5986-42dd-943d-e812a542b1f0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:29.371516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 867/1761] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Jun 24 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:29.373026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc7b2e-ab0b-4193-b587-47dd425f4a7a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:29.377888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:29.389365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc7b2e-ab0b-4193-b587-47dd425f4a7a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:29.390183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:29.390609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:29.390916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:29.395884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:29.399184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:29.399586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:29.400108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:29.417119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:29.422298 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:29.426490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:29.428126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:29.467446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 895/1762] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:29.477492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:29.479589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:29.483554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 868/1763] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:29.527083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f985e9-67c1-4343-8d4f-9017c2c89f67 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:29.539051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:29.539981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:29.540352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:29.542325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:29.590599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:29.598733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:29.609805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:29.613134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36c1c7-6427-4310-96a0-11f32978784f service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:29.641499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:29.642216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:29.642522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:29.642817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:29.660302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 896/1764] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:29.661704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:29.705975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:29.709549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:29.728792 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 869/1765] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:29.732997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c14cd-3159-47c9-bbbc-45095ac79826 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:29.753513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:29.754483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:29.754771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:29.755050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:29.790155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:29.793114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:29.810100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 897/1766] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:31.804528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:31.806260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:31.854185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ed957-b3dc-4f90-9f0a-4642136dab00 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:31.875420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:31.877829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:31.880838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:31.881162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] PATH_INFO: `/v3/projects/c84371bb67474ccabf04d10c9fc6f2ea` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:31.889773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84371bb67474ccabf04d10c9fc6f2ea)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:31.894146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:32.107616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84371bb67474ccabf04d10c9fc6f2ea', 'request_id': 'req-ba7b1842-4e7d-4480-b43f-cd6c275599dd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.127077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84371bb67474ccabf04d10c9fc6f2ea', 'request_id': 'req-ba7b1842-4e7d-4480-b43f-cd6c275599dd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.147274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84371bb67474ccabf04d10c9fc6f2ea', 'request_id': 'req-ba7b1842-4e7d-4480-b43f-cd6c275599dd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.203013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] The token cache is being invalidate because project c84371bb67474ccabf04d10c9fc6f2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:32.203449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.204858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.210346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 870/1767] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:31 2022] DELETE /identity/v3/projects/c84371bb67474ccabf04d10c9fc6f2ea => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:32.215228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:32.235942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3870-fadb-403a-9fef-663339d18f81 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:32.250250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:32.253877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:32.433587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:32.468224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b1842-4e7d-4480-b43f-cd6c275599dd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:32.475252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:32.476078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:32.476596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:32.477308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] PATH_INFO: `/v3/projects/46f0ce65859947e9b3b1cbf4362d57da` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:32.479013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f0ce65859947e9b3b1cbf4362d57da)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:32.481572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:32.581693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:32.582619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:32.583004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:32.583339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] PATH_INFO: `/v3/users/4a526b45bfc4477ea3f113d7f04ecd3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:32.585677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a526b45bfc4477ea3f113d7f04ecd3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:32.595143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:32.610571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f0ce65859947e9b3b1cbf4362d57da', 'request_id': 'req-d1aa39c9-f368-4e28-98a9-7a24c82d711d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.641105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f0ce65859947e9b3b1cbf4362d57da', 'request_id': 'req-d1aa39c9-f368-4e28-98a9-7a24c82d711d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.656599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f0ce65859947e9b3b1cbf4362d57da', 'request_id': 'req-d1aa39c9-f368-4e28-98a9-7a24c82d711d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.674495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] The token cache is being invalidate because project 46f0ce65859947e9b3b1cbf4362d57da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:32.675443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.676294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.681352 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 871/1768] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:32 2022] DELETE /identity/v3/projects/46f0ce65859947e9b3b1cbf4362d57da => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:32.682303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:32.685019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:32.823365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a526b45bfc4477ea3f113d7f04ecd3d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.873287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a526b45bfc4477ea3f113d7f04ecd3d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.890790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa39c9-f368-4e28-98a9-7a24c82d711d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:32.902902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a526b45bfc4477ea3f113d7f04ecd3d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.904065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:32.905968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a526b45bfc4477ea3f113d7f04ecd3d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.906525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:32.907016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a526b45bfc4477ea3f113d7f04ecd3d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:32.908324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:32.909384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] PATH_INFO: `/v3/users/4e97a49cef234e9d9fe0d0fa914dfa3c` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:32.916793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e97a49cef234e9d9fe0d0fa914dfa3c)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:32.921825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:32.997128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 898/1769] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:32 2022] DELETE /identity/v3/users/4a526b45bfc4477ea3f113d7f04ecd3d => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:32.999274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:33.009158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:33.052880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:33.106561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e97a49cef234e9d9fe0d0fa914dfa3c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.121818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e97a49cef234e9d9fe0d0fa914dfa3c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.140445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e97a49cef234e9d9fe0d0fa914dfa3c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.141994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e97a49cef234e9d9fe0d0fa914dfa3c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.142834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3887338b-f040-4574-a02d-2143fc0ccfe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e97a49cef234e9d9fe0d0fa914dfa3c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.168825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:33.190654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 872/1770] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:32 2022] DELETE /identity/v3/users/4e97a49cef234e9d9fe0d0fa914dfa3c => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:33.192367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33b28-881a-4c49-89fd-4f03300f53dd None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:33.192367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33b28-881a-4c49-89fd-4f03300f53dd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:33.192367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33b28-881a-4c49-89fd-4f03300f53dd None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:33.206042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 873/1771] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:19:33 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:33.231628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f490c-b1b5-4b94-9810-fe3ef4dfddb8 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:33.232183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f490c-b1b5-4b94-9810-fe3ef4dfddb8 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:33.232651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f490c-b1b5-4b94-9810-fe3ef4dfddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:33.256702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5572e3-6f46-4c83-8b6a-655564ed1987 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:33.313637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5d7f490c-b1b5-4b94-9810-fe3ef4dfddb8 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:33.319913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5d7f490c-b1b5-4b94-9810-fe3ef4dfddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:33.466977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:33.468595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:33.469157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:33.470153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] PATH_INFO: `/v3/users/dd93915267ab466fb60c1ff14b99c16d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:33.473733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd93915267ab466fb60c1ff14b99c16d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:33.484811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:33.592990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 874/1772] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:33.706670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd93915267ab466fb60c1ff14b99c16d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.727872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd93915267ab466fb60c1ff14b99c16d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.750847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd93915267ab466fb60c1ff14b99c16d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.752621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd93915267ab466fb60c1ff14b99c16d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.758266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd93915267ab466fb60c1ff14b99c16d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:33.801167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 899/1773] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:32 2022] DELETE /identity/v3/users/dd93915267ab466fb60c1ff14b99c16d => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:33.802953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:33.803389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:33.804057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:33.855253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:33.861555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:33.999188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 875/1774] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:34.026712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:34.029152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:34.216764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:34.298499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a110368-ecf9-45d8-9a7a-9391acc833f1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:34.324812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:34.324812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:34.324812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:34.324812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:34.359280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:34.371342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:34.378707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:34.380577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:34.404827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 900/1775] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:34.632942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00b8c3-b547-4d6a-82fb-c129a82e9444 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:34.653247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192998-b8a3-46c8-927d-c2ac85feebed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:34.657327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192998-b8a3-46c8-927d-c2ac85feebed service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:34.659230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192998-b8a3-46c8-927d-c2ac85feebed service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:34.659709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192998-b8a3-46c8-927d-c2ac85feebed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:34.668579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d8192998-b8a3-46c8-927d-c2ac85feebed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:34.689405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8192998-b8a3-46c8-927d-c2ac85feebed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:34.691658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8192998-b8a3-46c8-927d-c2ac85feebed service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:34.710429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 876/1776] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:35.541047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:35.543340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:35.560730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f6488-10d5-4bb6-92c0-fd78c89b7329 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:35.568612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:35.569522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:35.569986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:35.570588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] PATH_INFO: `/v3/projects/cc2cbb23e13849f381574dfecb9aee38` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:35.572385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc2cbb23e13849f381574dfecb9aee38)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:35.575193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:35.677724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc2cbb23e13849f381574dfecb9aee38', 'request_id': 'req-9048753c-c5c7-44b4-bbd9-0278e26c76b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:35.701619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc2cbb23e13849f381574dfecb9aee38', 'request_id': 'req-9048753c-c5c7-44b4-bbd9-0278e26c76b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:35.714635 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389fec3-30ad-4ca6-b883-07eb9467fa47 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:35.715151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389fec3-30ad-4ca6-b883-07eb9467fa47 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:35.715151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389fec3-30ad-4ca6-b883-07eb9467fa47 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:35.716964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 877/1777] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:35.719981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc2cbb23e13849f381574dfecb9aee38', 'request_id': 'req-9048753c-c5c7-44b4-bbd9-0278e26c76b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:35.739202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:35.739805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:35.743713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:35.752119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] The token cache is being invalidate because project cc2cbb23e13849f381574dfecb9aee38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:35.752529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:35.753857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9048753c-c5c7-44b4-bbd9-0278e26c76b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:35.757726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 901/1778] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:35 2022] DELETE /identity/v3/projects/cc2cbb23e13849f381574dfecb9aee38 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:35.788477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fdee9-a4c6-451b-bd3f-414e87da0685 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:35.795485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fdee9-a4c6-451b-bd3f-414e87da0685 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:35.796496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fdee9-a4c6-451b-bd3f-414e87da0685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:35.835136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:35.836815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:35.862076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:35.881854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-4f7fdee9-a4c6-451b-bd3f-414e87da0685 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:35.885224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4f7fdee9-a4c6-451b-bd3f-414e87da0685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.019001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 878/1779] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:36.032459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.042633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.108040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 902/1780] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:36.113735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090a23-b09d-44e8-ab2d-f2fb25ae1de1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.114197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090a23-b09d-44e8-ab2d-f2fb25ae1de1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.116015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090a23-b09d-44e8-ab2d-f2fb25ae1de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.166348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1b090a23-b09d-44e8-ab2d-f2fb25ae1de1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:36.170006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1b090a23-b09d-44e8-ab2d-f2fb25ae1de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.204098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 903/1781] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:36.210829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b090a23-b09d-44e8-ab2d-f2fb25ae1de1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.235749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b090a23-b09d-44e8-ab2d-f2fb25ae1de1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:36.251249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:36.253300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.253741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.254190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.255125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:36.255924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87866741-6e6d-44cb-adc5-e6942b0a3718 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:36.256404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:36.261944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:36.272880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:36.275221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.275508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.275790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] PATH_INFO: `/v3/projects/99ee61e24eb6418bbc51400af6e15066` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.277405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ee61e24eb6418bbc51400af6e15066)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:36.282192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 904/1782] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:36.282192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.282192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.285688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:36.298292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64b3af-bab4-4cfb-b94e-3fa67b503a64 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:36.316114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:36.317061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.317892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.318176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] PATH_INFO: `/v3/projects/05d6314a250b4e83b8ad1768b4ae3c89` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.319469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d6314a250b4e83b8ad1768b4ae3c89)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:36.321643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:36.420401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:36.437776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d6314a250b4e83b8ad1768b4ae3c89', 'request_id': 'req-65177c9d-3b85-422e-be9c-d13e564346c8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.455780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d6314a250b4e83b8ad1768b4ae3c89', 'request_id': 'req-65177c9d-3b85-422e-be9c-d13e564346c8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.457300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ee61e24eb6418bbc51400af6e15066', 'request_id': 'req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.464173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d6314a250b4e83b8ad1768b4ae3c89', 'request_id': 'req-65177c9d-3b85-422e-be9c-d13e564346c8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.473168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ee61e24eb6418bbc51400af6e15066', 'request_id': 'req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.482314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ee61e24eb6418bbc51400af6e15066', 'request_id': 'req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.485795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] The token cache is being invalidate because project 05d6314a250b4e83b8ad1768b4ae3c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:36.485795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.486929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.490383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 905/1783] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:36 2022] DELETE /identity/v3/projects/05d6314a250b4e83b8ad1768b4ae3c89 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:36.491153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.492498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.505055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] The token cache is being invalidate because project 99ee61e24eb6418bbc51400af6e15066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:36.505364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.506139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.508009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 879/1784] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:36 2022] DELETE /identity/v3/projects/99ee61e24eb6418bbc51400af6e15066 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:36.508999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.510394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.570371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65177c9d-3b85-422e-be9c-d13e564346c8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:36.571432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:36.624884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d12eee-6dd9-4f98-b83d-c3cf692f3b11 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:36.632129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:36.633043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.633398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.633919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.636356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.648396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908e227-035c-40dd-bcee-fa0bf34b4b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:36.649128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e227-035c-40dd-bcee-fa0bf34b4b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.649407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e227-035c-40dd-bcee-fa0bf34b4b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.649683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e227-035c-40dd-bcee-fa0bf34b4b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.650388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908e227-035c-40dd-bcee-fa0bf34b4b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:36.651283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:36.652316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908e227-035c-40dd-bcee-fa0bf34b4b1f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:36.653180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:36.667737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 880/1785] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:36.682667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.685916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.691097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 906/1786] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:36 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:36.691508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da97bb-c70e-4163-824c-d67222849631 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.691855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da97bb-c70e-4163-824c-d67222849631 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.692237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da97bb-c70e-4163-824c-d67222849631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.706017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d5ca2-c517-45c7-a267-13daf5844683 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:36.718453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-26da97bb-c70e-4163-824c-d67222849631 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:36.721077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-26da97bb-c70e-4163-824c-d67222849631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.779942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:36.782698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:36.784262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.784950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.785232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] PATH_INFO: `/v3/users/518dab278e0a49538b7d12024de76748` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.786766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=518dab278e0a49538b7d12024de76748)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:36.788932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:36.809097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 907/1787] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:36.810378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da97bb-c70e-4163-824c-d67222849631 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.814203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-26da97bb-c70e-4163-824c-d67222849631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:36.869033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da97bb-c70e-4163-824c-d67222849631 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:36.883609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b70c26-2590-430f-a845-357aa1259dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:36.886058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b70c26-2590-430f-a845-357aa1259dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.886058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b70c26-2590-430f-a845-357aa1259dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.886307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b70c26-2590-430f-a845-357aa1259dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.887633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b70c26-2590-430f-a845-357aa1259dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:36.890774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b70c26-2590-430f-a845-357aa1259dc8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:36.919722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518dab278e0a49538b7d12024de76748'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.935087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518dab278e0a49538b7d12024de76748'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.939214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 908/1788] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:36 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:36.943065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e913-757f-4693-814b-efd4f3cdcac4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:36.943779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e913-757f-4693-814b-efd4f3cdcac4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:36.944422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e913-757f-4693-814b-efd4f3cdcac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:36.945378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518dab278e0a49538b7d12024de76748'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.946422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518dab278e0a49538b7d12024de76748'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.947214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518dab278e0a49538b7d12024de76748'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:36.979303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 881/1789] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:36 2022] DELETE /identity/v3/users/518dab278e0a49538b7d12024de76748 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:36.981436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:36.984742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:37.019367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ea37e913-757f-4693-814b-efd4f3cdcac4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:37.027348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ea37e913-757f-4693-814b-efd4f3cdcac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:37.072274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:37.156188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0228c-206e-4260-aae0-4453c054da34 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.169789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.172911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.173237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.177166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.177166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.177700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 909/1790] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:37.178717 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e913-757f-4693-814b-efd4f3cdcac4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.181943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.207112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e913-757f-4693-814b-efd4f3cdcac4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.212267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 882/1791] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:37.217647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.227828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.232190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.232190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.232442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.237585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:37.237970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.240117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.249239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3c60-eda9-4a46-86ce-74f650dd479d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.264941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.266010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.266668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.267145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.270485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.273866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.281329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 910/1792] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:37.283420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.306967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34328838-9e6d-43c3-8bdc-bcf25c88eadd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.312907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 883/1793] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:37.317908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.320193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.321750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.322238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.328752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.333241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.341672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.372693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65447b4-48d9-45b8-913d-10516c5deb4e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.386536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02caf7-e1bb-44cc-ada1-d0204cae0788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.389053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02caf7-e1bb-44cc-ada1-d0204cae0788 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.389656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02caf7-e1bb-44cc-ada1-d0204cae0788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.390175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02caf7-e1bb-44cc-ada1-d0204cae0788 admin admin] PATH_INFO: `/v3/projects/eabe5ae95dae435e9557a4a9897d184c/users/7c3eee13db7b4491b9539f7266def66b/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.443893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02caf7-e1bb-44cc-ada1-d0204cae0788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eabe5ae95dae435e9557a4a9897d184c, user_id=7c3eee13db7b4491b9539f7266def66b, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.445662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 911/1794] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:37 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:37.452243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.456301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02caf7-e1bb-44cc-ada1-d0204cae0788 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.489201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 884/1795] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:37 2022] PUT /identity/v3/projects/eabe5ae95dae435e9557a4a9897d184c/users/7c3eee13db7b4491b9539f7266def66b/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:37.514135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99204298-7f50-41e2-983a-ac68b0a37209 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.517331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99204298-7f50-41e2-983a-ac68b0a37209 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.521006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99204298-7f50-41e2-983a-ac68b0a37209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.577777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-99204298-7f50-41e2-983a-ac68b0a37209 None None] MFA Rules not processed for user `7c3eee13db7b4491b9539f7266def66b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:37.579441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-99204298-7f50-41e2-983a-ac68b0a37209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:37.630049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46684b63-57d9-49ec-bb43-af19fc5ae581 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.678119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 885/1796] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:37.683161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99204298-7f50-41e2-983a-ac68b0a37209 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.717015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:37.738552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.741990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.742551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.743074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.744172 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.747203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.783002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 912/1797] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:37 2022] POST /identity/v3/users => generated 290 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:37.784373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.796746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f6738-d346-4122-83f9-abfdb4051ee5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.806956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99204298-7f50-41e2-983a-ac68b0a37209 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.817275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.818226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.818787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.819162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.820937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.828537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.830530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.830983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.831497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.832404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.833463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.834692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.849387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 913/1798] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:37.854505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.867915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c384a7-5c42-49d8-968d-ce286bed9b25 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.873662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.874891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.875332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.875748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.876752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.878642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:37.889741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 886/1799] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:37 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:37.891520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.895954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 914/1800] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:37.901235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:37.919158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dcbb9-31ec-4631-8982-eeaabb13e4ab admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.927075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:37.978233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29097ee0-a11a-4e52-96ef-cf74b6ab5260 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:37.983353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:37.984028 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:37.984311 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:37.984678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:37.985516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:37.988036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.014123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 887/1801] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:37 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:38.014430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d4574-2f73-40e0-ae28-a8d66debb3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.014430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d4574-2f73-40e0-ae28-a8d66debb3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.014430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d4574-2f73-40e0-ae28-a8d66debb3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.014430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d4574-2f73-40e0-ae28-a8d66debb3a7 admin admin] PATH_INFO: `/v3/projects/20972a0828c1453598f2b65179fc5804/users/e3dbfa7ce14346b18ae4963b600747e3/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.017137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.035418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a85da-82ce-408d-8d3f-fda5d2a130d4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.042751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.043433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.043771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.043963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.044749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.046333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.047446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d4574-2f73-40e0-ae28-a8d66debb3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20972a0828c1453598f2b65179fc5804, user_id=e3dbfa7ce14346b18ae4963b600747e3, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.051640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d4574-2f73-40e0-ae28-a8d66debb3a7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.061261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 888/1802] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:38.067245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.077204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 915/1803] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:37 2022] PUT /identity/v3/projects/20972a0828c1453598f2b65179fc5804/users/e3dbfa7ce14346b18ae4963b600747e3/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:38.088366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f558c-fe44-48cc-bc25-2057753d0395 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.088974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f558c-fe44-48cc-bc25-2057753d0395 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.089364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f558c-fe44-48cc-bc25-2057753d0395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.089487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a671c0b-baa6-42a3-94bf-9913ea772b1c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.129678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-915f558c-fe44-48cc-bc25-2057753d0395 None None] MFA Rules not processed for user `e3dbfa7ce14346b18ae4963b600747e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:38.132154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-915f558c-fe44-48cc-bc25-2057753d0395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:38.190248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.191946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.192528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.193113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] PATH_INFO: `/v3/projects/034e1f8fda584675b448a859dd27c27c/users/ceb8b2c863d442729a7fef08a5c4836c/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.221333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=034e1f8fda584675b448a859dd27c27c, user_id=ceb8b2c863d442729a7fef08a5c4836c, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.224248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.249499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 916/1804] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:38.253380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 889/1805] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:38 2022] PUT /identity/v3/projects/034e1f8fda584675b448a859dd27c27c/users/ceb8b2c863d442729a7fef08a5c4836c/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:38.253548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f558c-fe44-48cc-bc25-2057753d0395 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.256930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.274927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f558c-fe44-48cc-bc25-2057753d0395 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.327039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:38.367349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d726bec-9f80-47b0-9096-9032fbc49394 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.377098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.378279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.378691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.379091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.380380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.385247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.399663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 890/1806] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:38.401502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.407471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.408860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.409210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.409553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.410322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.412316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.417279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:38.435079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525808-c92d-45a0-b377-538e59465031 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.446314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.446314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.446314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.447183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] PATH_INFO: `/v3/projects/3962d4b1729d48bf8bac4a4273288601` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.449605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3962d4b1729d48bf8bac4a4273288601)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.452612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.471149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 917/1807] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:38 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:38.473021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.474662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:38.525279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3962d4b1729d48bf8bac4a4273288601', 'request_id': 'req-0ddc76a9-010b-475b-b523-9a7551df2b91'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:38.536579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3962d4b1729d48bf8bac4a4273288601', 'request_id': 'req-0ddc76a9-010b-475b-b523-9a7551df2b91'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:38.545588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3962d4b1729d48bf8bac4a4273288601', 'request_id': 'req-0ddc76a9-010b-475b-b523-9a7551df2b91'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:38.570095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] The token cache is being invalidate because project 3962d4b1729d48bf8bac4a4273288601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:38.570463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:38.573061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:38.578925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 891/1808] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:38 2022] DELETE /identity/v3/projects/3962d4b1729d48bf8bac4a4273288601 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:38.580392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.582491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:38.598311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21359357-cdf8-4f9e-bc44-09108d40495d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.671973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc76a9-010b-475b-b523-9a7551df2b91 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.677295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d07dc8-c9b1-4a43-8b4a-01f3d0baa4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.677847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d07dc8-c9b1-4a43-8b4a-01f3d0baa4fb admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.678347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d07dc8-c9b1-4a43-8b4a-01f3d0baa4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.678463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d07dc8-c9b1-4a43-8b4a-01f3d0baa4fb admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.679360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d07dc8-c9b1-4a43-8b4a-01f3d0baa4fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.679819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a7e83-a929-40fc-b5ff-c30d771d4c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.680876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a7e83-a929-40fc-b5ff-c30d771d4c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.681250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a7e83-a929-40fc-b5ff-c30d771d4c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.681355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d07dc8-c9b1-4a43-8b4a-01f3d0baa4fb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.681608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a7e83-a929-40fc-b5ff-c30d771d4c83 admin admin] PATH_INFO: `/v3/projects/034e1f8fda584675b448a859dd27c27c/users/ceb8b2c863d442729a7fef08a5c4836c/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.684446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a7e83-a929-40fc-b5ff-c30d771d4c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=034e1f8fda584675b448a859dd27c27c, user_id=ceb8b2c863d442729a7fef08a5c4836c, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.687708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a7e83-a929-40fc-b5ff-c30d771d4c83 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.708769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 918/1809] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:38 2022] PUT /identity/v3/projects/034e1f8fda584675b448a859dd27c27c/users/ceb8b2c863d442729a7fef08a5c4836c/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:38.710602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c978b7-ee7a-4908-9666-caa2efc8e5e8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.710901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c978b7-ee7a-4908-9666-caa2efc8e5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.711183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c978b7-ee7a-4908-9666-caa2efc8e5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.716779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 892/1810] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:38 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:38.717363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c65b7-0285-461b-ba8c-6999c14d2766 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.718092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c65b7-0285-461b-ba8c-6999c14d2766 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.718092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c65b7-0285-461b-ba8c-6999c14d2766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.744090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-59c978b7-ee7a-4908-9666-caa2efc8e5e8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:38.745993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-59c978b7-ee7a-4908-9666-caa2efc8e5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:38.754545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-2e2c65b7-0285-461b-ba8c-6999c14d2766 None None] MFA Rules not processed for user `ceb8b2c863d442729a7fef08a5c4836c`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:38.755760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2e2c65b7-0285-461b-ba8c-6999c14d2766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:38.823058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:38.840596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 919/1811] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:38.844592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c978b7-ee7a-4908-9666-caa2efc8e5e8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.858380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c978b7-ee7a-4908-9666-caa2efc8e5e8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.864727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.865515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.866029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.866392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.867386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.869115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.873214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 893/1812] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:38.874763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90284-e6db-48b8-9e26-d502cb7c9465 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.875021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90284-e6db-48b8-9e26-d502cb7c9465 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.875270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc90284-e6db-48b8-9e26-d502cb7c9465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.884517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 920/1813] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:38.885541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.887683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:38.903780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7bc90284-e6db-48b8-9e26-d502cb7c9465 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:38.905412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7bc90284-e6db-48b8-9e26-d502cb7c9465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:38.915364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 894/1814] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:38.919966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc90284-e6db-48b8-9e26-d502cb7c9465 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.931875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc90284-e6db-48b8-9e26-d502cb7c9465 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.940762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.941780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.942076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.942355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] PATH_INFO: `/v3/projects/f275eb37b0214305ade32482a39ee658/users/26348282fd8c44cf92d242c374558f74/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.947668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:38.959566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f645e-7ba7-4daa-bc45-b1843ad3743b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:38.964845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:38.966019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:38.966492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:38.966815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:38.969491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f275eb37b0214305ade32482a39ee658, user_id=26348282fd8c44cf92d242c374558f74, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.974020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.979612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:38.981353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:38.994409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 895/1815] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:38 2022] PUT /identity/v3/projects/f275eb37b0214305ade32482a39ee658/users/26348282fd8c44cf92d242c374558f74/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:38.994409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:38.994846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 921/1816] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:38.995791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.009457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012cae-c205-4a74-8254-1139f32b3231 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.094069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.094761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.095075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.095890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.096942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.098853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.114446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 922/1817] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:39.115361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.117866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:39.135055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbcf96-ae33-450e-9beb-3e66235ed704 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.137132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a4138-c45c-470e-a3cd-064a761251d9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.141154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.141840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.142126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.142463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.143214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:39.143530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.143640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.144414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.144893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.145163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] PATH_INFO: `/v3/users/7c3eee13db7b4491b9539f7266def66b` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.145272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.147276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3eee13db7b4491b9539f7266def66b)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.150553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.161449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 896/1818] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:39.162634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.179922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3958bd-b3df-417d-8308-954736d1ac49 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.185713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.186627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.186958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.187858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] PATH_INFO: `/v3/projects/f275eb37b0214305ade32482a39ee658/users/26348282fd8c44cf92d242c374558f74/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.190701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f275eb37b0214305ade32482a39ee658, user_id=26348282fd8c44cf92d242c374558f74, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.193638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.216426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 897/1819] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:39 2022] PUT /identity/v3/projects/f275eb37b0214305ade32482a39ee658/users/26348282fd8c44cf92d242c374558f74/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:39.217557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.219691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3eee13db7b4491b9539f7266def66b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:39.237242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3eee13db7b4491b9539f7266def66b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:39.251489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3eee13db7b4491b9539f7266def66b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:39.258572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3eee13db7b4491b9539f7266def66b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:39.261769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8eddc2e8-3706-4937-a70f-3d9b8fb46985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3eee13db7b4491b9539f7266def66b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:39.304946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:39.311425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de2f90-fa72-406b-a856-31fe5f52ace5 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.319114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b6ae3-1e4f-4d77-8d8b-a1b9545ddd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.319877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6ae3-1e4f-4d77-8d8b-a1b9545ddd6a admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.320209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6ae3-1e4f-4d77-8d8b-a1b9545ddd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.320371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 923/1820] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:39 2022] DELETE /identity/v3/users/7c3eee13db7b4491b9539f7266def66b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:39.321829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.323471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6ae3-1e4f-4d77-8d8b-a1b9545ddd6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.323613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.323886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.324228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b6ae3-1e4f-4d77-8d8b-a1b9545ddd6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.327021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b6ae3-1e4f-4d77-8d8b-a1b9545ddd6a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.369802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 898/1821] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:39 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:39.369802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.369802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.369802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.382276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] MFA Rules not processed for user `26348282fd8c44cf92d242c374558f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:39.383673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:39.397796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:39.400663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:39.497397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:39.513237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 924/1822] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:39.513911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.517032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:39.548448 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 899/1823] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:39.557917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.565029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:39.674907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b96f5-57ad-45d4-b180-00fc6ab1eff0 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.682374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.684444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.685041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.685517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.686604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.689416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.712448 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa232c6-2a09-4d56-8ba6-a82b8161385b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.720271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.722661 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.722952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.723237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.724212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 925/1824] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:39 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:39.725254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.728773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:39.746962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.751424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.781497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 900/1825] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:39.782402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.802742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceecfd3-ed2e-418f-b3c2-5b6fe273633e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.805766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158301b-57f8-44af-87e4-24180e15eb01 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.811107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.811786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.812065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.812340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.812738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.813279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.813923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.814422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.814913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.815118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.826269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 901/1826] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:39.831533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.833290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.836202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.846515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d280e-8aa7-42ae-abf9-46a0c967e5c4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.852109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69463949-69d9-46c0-a492-71eaea4f49b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.852865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69463949-69d9-46c0-a492-71eaea4f49b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.853518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69463949-69d9-46c0-a492-71eaea4f49b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.853871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69463949-69d9-46c0-a492-71eaea4f49b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.854822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69463949-69d9-46c0-a492-71eaea4f49b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.857428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69463949-69d9-46c0-a492-71eaea4f49b7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.865835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 926/1827] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:39.871166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 902/1828] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:39.876276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:39.895018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630da388-8493-45b0-98ae-100ed5b6226e service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:39.904759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:39.905674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.906061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.906233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] PATH_INFO: `/v3/projects/57d971df6ca64a209bbecf001aea602f/users/9e78d07a02c34abcafa7ecd57a8ff1d2/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:39.926447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:39.931847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d971df6ca64a209bbecf001aea602f, user_id=9e78d07a02c34abcafa7ecd57a8ff1d2, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:39.935223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:39.955911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 927/1829] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:39 2022] PUT /identity/v3/projects/57d971df6ca64a209bbecf001aea602f/users/9e78d07a02c34abcafa7ecd57a8ff1d2/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:39.969680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:39.970214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:39.971130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:40.001634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] MFA Rules not processed for user `9e78d07a02c34abcafa7ecd57a8ff1d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:40.002799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:40.114738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 903/1830] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:40.119002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:40.237250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7f321-b836-4ace-a320-62d6a88777e1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:40.244748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:40.245432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:40.245725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:40.246629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:40.247769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:40.252178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:40.253655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:40.259051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:40.284331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d714b0b-3eac-4897-b21e-70283699a39b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:40.296713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:40.298418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:40.299648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:40.300320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] PATH_INFO: `/v3/projects/eabe5ae95dae435e9557a4a9897d184c` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:40.303062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eabe5ae95dae435e9557a4a9897d184c)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:40.309081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 928/1831] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:40 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:40.309081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:40.309350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:40.414697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eabe5ae95dae435e9557a4a9897d184c', 'request_id': 'req-d10932c9-94c1-492e-8e9d-5be79cf9564a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.420432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:40.428484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eabe5ae95dae435e9557a4a9897d184c', 'request_id': 'req-d10932c9-94c1-492e-8e9d-5be79cf9564a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.429863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fb4ea-4bb0-42ea-9ea3-edd9098647cc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:40.444924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eabe5ae95dae435e9557a4a9897d184c', 'request_id': 'req-d10932c9-94c1-492e-8e9d-5be79cf9564a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.462644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] The token cache is being invalidate because project eabe5ae95dae435e9557a4a9897d184c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:40.462844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.463643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.468095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 904/1832] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:40 2022] DELETE /identity/v3/projects/eabe5ae95dae435e9557a4a9897d184c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:40.468095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:40.485576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:40.553176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:40.554419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:40.554774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:40.555128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:40.555927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:40.558146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:40.591631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 929/1833] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:40 2022] POST /identity/v3/users => generated 295 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:40.592804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:40.595496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:40.609362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf0af1-6232-491d-95c7-4930d5b376ce admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:40.615218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:40.616300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:40.616688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:40.617194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] PATH_INFO: `/v3/users/ceb8b2c863d442729a7fef08a5c4836c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:40.618832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb8b2c863d442729a7fef08a5c4836c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:40.621040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:40.621456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10932c9-94c1-492e-8e9d-5be79cf9564a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:40.626412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:40.627840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:40.628124 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:40.629118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] PATH_INFO: `/v3/users/e3dbfa7ce14346b18ae4963b600747e3` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:40.630159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3dbfa7ce14346b18ae4963b600747e3)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:40.630930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:40.631834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:40.709378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb8b2c863d442729a7fef08a5c4836c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.719911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb8b2c863d442729a7fef08a5c4836c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.724239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3dbfa7ce14346b18ae4963b600747e3'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.730402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb8b2c863d442729a7fef08a5c4836c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.731531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb8b2c863d442729a7fef08a5c4836c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.732405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb8b2c863d442729a7fef08a5c4836c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.758237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3dbfa7ce14346b18ae4963b600747e3'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.771133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 930/1834] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:40 2022] DELETE /identity/v3/users/ceb8b2c863d442729a7fef08a5c4836c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:40.772002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:40.774113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:40.780918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3dbfa7ce14346b18ae4963b600747e3'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.781882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3dbfa7ce14346b18ae4963b600747e3'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.782664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88b40ecf-154f-4b9b-b22a-303f7c7b70d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3dbfa7ce14346b18ae4963b600747e3'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:40.828728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 905/1835] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:40 2022] DELETE /identity/v3/users/e3dbfa7ce14346b18ae4963b600747e3 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:40.833705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:40.834432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:40.834916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:40.874434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:40.879743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:40.891135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996815-b001-492b-9f68-098366e4e6fa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:41.000662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:41.001738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.002487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.002947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.003848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:41.005164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 906/1836] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:41.006650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:41.017531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 931/1837] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:41.027647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:41.029189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:41.044299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:41.044700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:41.050385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:41.055853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb293-17f8-4c4f-b4e3-3fa5400add16 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:41.065045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:41.066841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.067337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.067782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] PATH_INFO: `/v3/projects/4aedf5a132df4ffd9bfa8be445e008fa/users/ce7f0426b0c947e2a8e4c838c0be7f24/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.093380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aedf5a132df4ffd9bfa8be445e008fa, user_id=ce7f0426b0c947e2a8e4c838c0be7f24, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:41.097337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:41.097910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:41.121174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 907/1838] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:41 2022] PUT /identity/v3/projects/4aedf5a132df4ffd9bfa8be445e008fa/users/ce7f0426b0c947e2a8e4c838c0be7f24/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:41.127843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:41.147328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72709f-0492-436a-93ef-37a44949a9ad admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:41.235875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291edec-e0d3-4d76-bb1d-48c1629107a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:41.236586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291edec-e0d3-4d76-bb1d-48c1629107a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.236943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291edec-e0d3-4d76-bb1d-48c1629107a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.237164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291edec-e0d3-4d76-bb1d-48c1629107a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.247660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9877f6-a9ed-4f48-bbb0-93fec9a31475 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:41.255771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:41.256230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291edec-e0d3-4d76-bb1d-48c1629107a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:41.256435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.256719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.256996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.257782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:41.259387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:41.259516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291edec-e0d3-4d76-bb1d-48c1629107a8 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:41.272494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 908/1839] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:41.273693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 932/1840] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:41.276327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:41.288745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060a76a-fbb5-47fe-855d-591582bb4545 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:41.301129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:41.302490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.302779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.303644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] PATH_INFO: `/v3/projects/4aedf5a132df4ffd9bfa8be445e008fa/users/ce7f0426b0c947e2a8e4c838c0be7f24/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.307854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aedf5a132df4ffd9bfa8be445e008fa, user_id=ce7f0426b0c947e2a8e4c838c0be7f24, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:41.311105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:41.324209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 909/1841] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:41 2022] PUT /identity/v3/projects/4aedf5a132df4ffd9bfa8be445e008fa/users/ce7f0426b0c947e2a8e4c838c0be7f24/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:41.335990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.336328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.336594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.379341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] MFA Rules not processed for user `ce7f0426b0c947e2a8e4c838c0be7f24`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:41.380648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:41.476260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 933/1842] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:41.502684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:41.502911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:41.504712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:41.505472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:41.617469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61e569-0aca-4b1e-b57c-6439222934f8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:41.621645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802a4fe-9115-4c70-95bb-2d767894bcc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:41.622486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a4fe-9115-4c70-95bb-2d767894bcc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.623564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a4fe-9115-4c70-95bb-2d767894bcc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.623855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802a4fe-9115-4c70-95bb-2d767894bcc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.625899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a802a4fe-9115-4c70-95bb-2d767894bcc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:41.643679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30489f16-337e-4c40-aeb2-239ceae79908 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:41.650649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:41.652000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.652315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.652869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] PATH_INFO: `/v3/projects/034e1f8fda584675b448a859dd27c27c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.654442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] RBAC: Authorizing `identity:delete_project(project_id=034e1f8fda584675b448a859dd27c27c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:41.656854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:41.703061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:41.799268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034e1f8fda584675b448a859dd27c27c', 'request_id': 'req-1f222035-9783-4172-b2d3-d91e1374334d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:41.809001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802a4fe-9115-4c70-95bb-2d767894bcc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:41.813425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034e1f8fda584675b448a859dd27c27c', 'request_id': 'req-1f222035-9783-4172-b2d3-d91e1374334d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:41.814528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802a4fe-9115-4c70-95bb-2d767894bcc3 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:41.825256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034e1f8fda584675b448a859dd27c27c', 'request_id': 'req-1f222035-9783-4172-b2d3-d91e1374334d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:41.838036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 910/1843] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:41.843280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] The token cache is being invalidate because project 034e1f8fda584675b448a859dd27c27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:41.844270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:41.844417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f222035-9783-4172-b2d3-d91e1374334d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:41.846850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 934/1844] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:41 2022] DELETE /identity/v3/projects/034e1f8fda584675b448a859dd27c27c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:41.852906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:41.855113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:41.855417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:41.889547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:41.891341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:41.905868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:42.018696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 911/1845] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:42.027180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe603-e965-401c-ad03-90e957e6eba4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.027736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe603-e965-401c-ad03-90e957e6eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.028153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe603-e965-401c-ad03-90e957e6eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.082127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.084955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:42.086929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-27cfe603-e965-401c-ad03-90e957e6eba4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:42.089723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-27cfe603-e965-401c-ad03-90e957e6eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:42.104466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c9c42-186a-4441-96db-36d4f36fab4a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.112946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 935/1846] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:42.118215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.118361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfe603-e965-401c-ad03-90e957e6eba4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.118976 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.119262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.119540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] PATH_INFO: `/v3/projects/20972a0828c1453598f2b65179fc5804` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.121310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20972a0828c1453598f2b65179fc5804)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.142995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.144058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfe603-e965-401c-ad03-90e957e6eba4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.154805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.154805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.154805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.155254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.158794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:42.158794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.161141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.186253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 936/1847] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:19:42.187228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.200521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef10d-a065-4615-a77e-031627d5fed0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.205997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.206671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.206957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.207281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.207979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.210532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.260339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 937/1848] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:42 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:42.264767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.274292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20972a0828c1453598f2b65179fc5804', 'request_id': 'req-89b1c85d-4575-4386-a570-13665a663029'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.284963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20972a0828c1453598f2b65179fc5804', 'request_id': 'req-89b1c85d-4575-4386-a570-13665a663029'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.293139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20972a0828c1453598f2b65179fc5804', 'request_id': 'req-89b1c85d-4575-4386-a570-13665a663029'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.309480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] The token cache is being invalidate because project 20972a0828c1453598f2b65179fc5804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:42.310083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.311430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.320070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 912/1849] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:42 2022] DELETE /identity/v3/projects/20972a0828c1453598f2b65179fc5804 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:42.321439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.323265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:42.395370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19fceb-ddef-44a3-8223-be658c9d999e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.436891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.438156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.438446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.438733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.440608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c85d-4575-4386-a570-13665a663029 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.445748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.446931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.447275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.447615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] PATH_INFO: `/v3/users/26348282fd8c44cf92d242c374558f74` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.448952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26348282fd8c44cf92d242c374558f74)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.450189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.451294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.453905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.482452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 938/1850] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:42 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:42.487988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.493038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:42.519260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14771bc2-38c3-4b37-86fd-edd17c394563 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.526375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.527057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.527379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.527863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.528671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.530629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:42.534068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.543738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26348282fd8c44cf92d242c374558f74'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.559490 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 939/1851] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:42.561052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.564784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26348282fd8c44cf92d242c374558f74'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.583130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26348282fd8c44cf92d242c374558f74'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.584197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26348282fd8c44cf92d242c374558f74'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.584898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26348282fd8c44cf92d242c374558f74'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:42.602767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6744a8c-0855-4a90-be90-42c7e1a206bf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.611972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff1776-4ef7-48a0-b76d-480d216798d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.613408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff1776-4ef7-48a0-b76d-480d216798d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.614112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff1776-4ef7-48a0-b76d-480d216798d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.614868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff1776-4ef7-48a0-b76d-480d216798d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.617964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff1776-4ef7-48a0-b76d-480d216798d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.621458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff1776-4ef7-48a0-b76d-480d216798d7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.627236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 913/1852] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:42 2022] DELETE /identity/v3/users/26348282fd8c44cf92d242c374558f74 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:42.643355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 940/1853] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:42.648263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:42.649947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:42.758132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e24fe-c2a7-40a1-981b-7933276bba18 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:42.763568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:42.764854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.765374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.765722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] PATH_INFO: `/v3/projects/c95c7335d30a4c4bb27e3f44a54e1792/users/dca46149fa0d4afb8ee40f76a278aa7a/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.785358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95c7335d30a4c4bb27e3f44a54e1792, user_id=dca46149fa0d4afb8ee40f76a278aa7a, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:42.788649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:42.804396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 914/1854] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:42 2022] PUT /identity/v3/projects/c95c7335d30a4c4bb27e3f44a54e1792/users/dca46149fa0d4afb8ee40f76a278aa7a/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:42.815848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f93af-4779-4944-ba63-e1abb41200d8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:42.816314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f93af-4779-4944-ba63-e1abb41200d8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:42.816468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f93af-4779-4944-ba63-e1abb41200d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:42.852933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ad0f93af-4779-4944-ba63-e1abb41200d8 None None] MFA Rules not processed for user `dca46149fa0d4afb8ee40f76a278aa7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:42.854190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ad0f93af-4779-4944-ba63-e1abb41200d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:42.873445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:42.944877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 941/1855] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:42.946091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:43.038795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6a631-89cd-4d9b-9f84-5141cad08cc8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:43.046502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:43.047683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:43.049021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:43.049486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:43.050358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:43.052834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:43.102833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 915/1856] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:42 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:43.104395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f93af-4779-4944-ba63-e1abb41200d8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:43.139938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f93af-4779-4944-ba63-e1abb41200d8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:43.280533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:43.281872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:43.282190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:43.283275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:43.284418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:43.286826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:43.346087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 942/1857] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:43 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:43.349914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:43.385659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988a19e-59e1-41a5-95d4-180b4239af3a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:43.402577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:43.403270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:43.403550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:43.403820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:43.404581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:43.406775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:43.432747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 916/1858] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:43.437453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:43.467400 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa32ba8-9d09-4dc5-b5ba-379c39733be7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:43.523196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:43.524130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:43.524411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:43.524720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] PATH_INFO: `/v3/projects/a66ce97b1caf4766b4804a370d881244/users/9394e5cb70774e70862494404db6997d/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:43.563748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66ce97b1caf4766b4804a370d881244, user_id=9394e5cb70774e70862494404db6997d, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:43.566696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:43.594833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 943/1859] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:43 2022] PUT /identity/v3/projects/a66ce97b1caf4766b4804a370d881244/users/9394e5cb70774e70862494404db6997d/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:43.599266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:43.613939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dd546-fb60-4537-8cc2-fb7df5c6aa62 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:43.629828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:43.728954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071200ca-c9f9-4082-b1dc-4c91f17b85a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:43.730547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071200ca-c9f9-4082-b1dc-4c91f17b85a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:43.730841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071200ca-c9f9-4082-b1dc-4c91f17b85a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:43.731120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071200ca-c9f9-4082-b1dc-4c91f17b85a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:43.733657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071200ca-c9f9-4082-b1dc-4c91f17b85a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:43.736408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071200ca-c9f9-4082-b1dc-4c91f17b85a8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:43.760122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 917/1860] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:43.762935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:43.775565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f60623-95b9-475b-8e86-10ada9d440df admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:43.781331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:43.782438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:43.783345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:43.783935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] PATH_INFO: `/v3/projects/a66ce97b1caf4766b4804a370d881244/users/9394e5cb70774e70862494404db6997d/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:43.786959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66ce97b1caf4766b4804a370d881244, user_id=9394e5cb70774e70862494404db6997d, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:43.790162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:43.814275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 944/1861] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:43 2022] PUT /identity/v3/projects/a66ce97b1caf4766b4804a370d881244/users/9394e5cb70774e70862494404db6997d/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:43.818667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:43.819236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:43.819740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:43.862860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] MFA Rules not processed for user `9394e5cb70774e70862494404db6997d`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:43.864275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:43.975739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:43.980020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:44.068992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 918/1862] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:44.085583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.087825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:44.089505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044209d7-e7ef-45f2-be4c-a51a31719661 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.096403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.097059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.097346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.097622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] PATH_INFO: `/v3/projects/f275eb37b0214305ade32482a39ee658` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.098870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f275eb37b0214305ade32482a39ee658)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.101598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.139980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:44.165170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf684-cffe-4d77-b6e7-12de109ea497 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.173411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0954be2-8f09-46df-a6e6-74dbb0d3121b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.174506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0954be2-8f09-46df-a6e6-74dbb0d3121b service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.174781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0954be2-8f09-46df-a6e6-74dbb0d3121b service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.175054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0954be2-8f09-46df-a6e6-74dbb0d3121b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.186296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0954be2-8f09-46df-a6e6-74dbb0d3121b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.187738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0954be2-8f09-46df-a6e6-74dbb0d3121b service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.204601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 919/1863] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:44.212214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f275eb37b0214305ade32482a39ee658', 'request_id': 'req-273f914b-0c11-400d-becf-78537a4b9e04'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:44.230336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f275eb37b0214305ade32482a39ee658', 'request_id': 'req-273f914b-0c11-400d-becf-78537a4b9e04'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:44.238719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f275eb37b0214305ade32482a39ee658', 'request_id': 'req-273f914b-0c11-400d-becf-78537a4b9e04'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:44.252469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] The token cache is being invalidate because project f275eb37b0214305ade32482a39ee658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:44.252882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:44.253474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-273f914b-0c11-400d-becf-78537a4b9e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:44.255284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 945/1864] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:43 2022] DELETE /identity/v3/projects/f275eb37b0214305ade32482a39ee658 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:44.259996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb7bb8-988a-456d-bc98-1f874b508e85 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.260302 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb7bb8-988a-456d-bc98-1f874b508e85 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.260650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb7bb8-988a-456d-bc98-1f874b508e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.279290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3eeb7bb8-988a-456d-bc98-1f874b508e85 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:44.280504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3eeb7bb8-988a-456d-bc98-1f874b508e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:44.320397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:44.351822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 920/1865] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:44.359636 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82adcb5-b6f6-4f56-a34c-53a9e0b1d140 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.359926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82adcb5-b6f6-4f56-a34c-53a9e0b1d140 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.360205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82adcb5-b6f6-4f56-a34c-53a9e0b1d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.379093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d82adcb5-b6f6-4f56-a34c-53a9e0b1d140 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:44.380284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d82adcb5-b6f6-4f56-a34c-53a9e0b1d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:44.388719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 946/1866] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:44.393060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb7bb8-988a-456d-bc98-1f874b508e85 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.402961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb7bb8-988a-456d-bc98-1f874b508e85 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.407707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.408388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.408729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.409095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.409913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:44.410244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.412189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.425757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 921/1867] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:19:44.430630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82adcb5-b6f6-4f56-a34c-53a9e0b1d140 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.441049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82adcb5-b6f6-4f56-a34c-53a9e0b1d140 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.446080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.446747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.447032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.447308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.448034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.450035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.488497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 947/1868] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:44 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:44.492571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.504592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3518e-d76f-48f8-9abf-54ff68562fd2 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.575129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.577064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:44.592125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.593142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.593427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.593705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.594478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.596983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.623596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 922/1869] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:44 2022] POST /identity/v3/users => generated 281 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:44.624866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.638025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df92c92-ba0c-4321-9752-b54ae97a9cbc admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.647434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.649172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.649464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.649757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.650527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.652204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.662405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4be3f-f611-4610-829b-c13fa1875d2c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.667146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9394e5cb70774e70862494404db6997d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a66ce97b1caf4766b4804a370d881244', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.667825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.668111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.668396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.669167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:44.669477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.674542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 923/1870] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:44.674834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.676063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.689977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c722e1-365d-42c9-98b7-c27a109ec197 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.690453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 948/1871] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:44.692164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.695097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.695766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.696045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.696325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.697106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.698690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.701878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c53a4e-181d-4ed1-a836-275d95aeb379 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.706502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9394e5cb70774e70862494404db6997d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a66ce97b1caf4766b4804a370d881244', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.707195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.707489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.707769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.708454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.712040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 924/1872] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:44.712189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.713058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.723438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901628b-8380-48b5-93dd-ff7a28f37069 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.728130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583e82e-79ee-4773-a3d5-fc917c8f5ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.729067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583e82e-79ee-4773-a3d5-fc917c8f5ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.729346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583e82e-79ee-4773-a3d5-fc917c8f5ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.729749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583e82e-79ee-4773-a3d5-fc917c8f5ae5 admin admin] PATH_INFO: `/v3/projects/fe0488c57a074a8893e4f483b3dd5ae8/users/7a10c555756e4a06b6c8a529ab85d27d/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.738748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 949/1873] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:44 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:44.742722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.746412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583e82e-79ee-4773-a3d5-fc917c8f5ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe0488c57a074a8893e4f483b3dd5ae8, user_id=7a10c555756e4a06b6c8a529ab85d27d, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.749483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583e82e-79ee-4773-a3d5-fc917c8f5ae5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.763785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 925/1874] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:44 2022] PUT /identity/v3/projects/fe0488c57a074a8893e4f483b3dd5ae8/users/7a10c555756e4a06b6c8a529ab85d27d/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:44.774945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.775228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.775502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.791199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:44.805218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] MFA Rules not processed for user `7a10c555756e4a06b6c8a529ab85d27d`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:44.806423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:44.810774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e42a9-69dd-444c-9ac5-6eef49d5bb75 tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.868379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9394e5cb70774e70862494404db6997d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a66ce97b1caf4766b4804a370d881244', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.868990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.869276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.869578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.869989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:44.870283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:44.870597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.872333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.874533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 926/1875] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:44.885338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 950/1876] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:19:44.908845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.910329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:44.922712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:44.933729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fd27a-66c6-4ba4-be1c-064721fa89de tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.938476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9394e5cb70774e70862494404db6997d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a66ce97b1caf4766b4804a370d881244', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.939414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.939787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.940076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] PATH_INFO: `/v3/projects/39122f5600294b02af9d3f4655bda57c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.941346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorizing `identity:delete_project(project_id=39122f5600294b02af9d3f4655bda57c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.943302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.972547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0c85-8902-4bf8-8f95-c98eb128da40 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:44.976956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:44.977773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:44.978057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:44.978328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:44.987424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:44.988882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:44.998608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 927/1877] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:45.007347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39122f5600294b02af9d3f4655bda57c', 'request_id': 'req-8306f8ec-49bf-44f6-be83-511e279b902d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.012270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:45.027968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39122f5600294b02af9d3f4655bda57c', 'request_id': 'req-8306f8ec-49bf-44f6-be83-511e279b902d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.045760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39122f5600294b02af9d3f4655bda57c', 'request_id': 'req-8306f8ec-49bf-44f6-be83-511e279b902d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.103897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2584c-a76c-44e6-9dd2-f74aaaf1c7ef service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:45.185293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:45.186181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:45.186471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:45.186752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] PATH_INFO: `/v3/users/7a10c555756e4a06b6c8a529ab85d27d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:45.187206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] The token cache is being invalidate because project 39122f5600294b02af9d3f4655bda57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:45.187503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.188039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a10c555756e4a06b6c8a529ab85d27d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:45.188213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-8306f8ec-49bf-44f6-be83-511e279b902d tempest-VolumeQuotaClassesTest-723559366 tempest-VolumeQuotaClassesTest-723559366-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.190079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:45.191227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 951/1878] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:44 2022] DELETE /identity/v3/projects/39122f5600294b02af9d3f4655bda57c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:45.266786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a10c555756e4a06b6c8a529ab85d27d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.275394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:45.280694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a10c555756e4a06b6c8a529ab85d27d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.297377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a10c555756e4a06b6c8a529ab85d27d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.298894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a10c555756e4a06b6c8a529ab85d27d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.299902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a10c555756e4a06b6c8a529ab85d27d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.337268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 928/1879] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:45 2022] DELETE /identity/v3/users/7a10c555756e4a06b6c8a529ab85d27d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:45.340740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:45.341121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:45.341779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:45.361217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:45.362623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:45.478026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 952/1880] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:45.488369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:45.489917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:45.492158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:45.494327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:45.513587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513df609-3128-4fb3-827f-5033a4d8dcf1 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:45.519499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:45.520474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:45.520860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:45.521200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] PATH_INFO: `/v3/users/dca46149fa0d4afb8ee40f76a278aa7a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:45.522507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca46149fa0d4afb8ee40f76a278aa7a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:45.524540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:45.527983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:45.568150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf1376-1941-45ac-bf49-60ff44ec25a0 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:45.573449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4483-f70f-4b94-be04-ee0c85b8fa62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:45.574399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4483-f70f-4b94-be04-ee0c85b8fa62 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:45.574776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4483-f70f-4b94-be04-ee0c85b8fa62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:45.575218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4483-f70f-4b94-be04-ee0c85b8fa62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:45.586577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d4483-f70f-4b94-be04-ee0c85b8fa62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:45.588023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d4483-f70f-4b94-be04-ee0c85b8fa62 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:45.588535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca46149fa0d4afb8ee40f76a278aa7a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.597258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca46149fa0d4afb8ee40f76a278aa7a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.599978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 929/1881] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:45.606314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca46149fa0d4afb8ee40f76a278aa7a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.607023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca46149fa0d4afb8ee40f76a278aa7a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.607675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca46149fa0d4afb8ee40f76a278aa7a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:45.625968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 953/1882] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:45 2022] DELETE /identity/v3/users/dca46149fa0d4afb8ee40f76a278aa7a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:45.631228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:45.631511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:45.631792 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:45.650680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:45.652053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:45.719361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 930/1883] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:45.731652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:45.733356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:45.810621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c018c-f266-4f92-aedf-b938dbbadc5b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:45.814773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704acc9-c306-496e-8daa-5afca53dbac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:45.815450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704acc9-c306-496e-8daa-5afca53dbac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:45.815739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704acc9-c306-496e-8daa-5afca53dbac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:45.816051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704acc9-c306-496e-8daa-5afca53dbac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:45.827639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704acc9-c306-496e-8daa-5afca53dbac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:45.830426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704acc9-c306-496e-8daa-5afca53dbac7 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:45.843775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 954/1884] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:45.961539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:45.963157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:45.977307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2259b-d5fe-4067-a669-34d62a115b54 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:45.983294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:45.984299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:45.984717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:45.985099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] PATH_INFO: `/v3/projects/fe0488c57a074a8893e4f483b3dd5ae8` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:45.986478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe0488c57a074a8893e4f483b3dd5ae8)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:45.988501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.046153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe0488c57a074a8893e4f483b3dd5ae8', 'request_id': 'req-9a33af8b-df8c-40d7-80a9-5e9896241875'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.056321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe0488c57a074a8893e4f483b3dd5ae8', 'request_id': 'req-9a33af8b-df8c-40d7-80a9-5e9896241875'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.067236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe0488c57a074a8893e4f483b3dd5ae8', 'request_id': 'req-9a33af8b-df8c-40d7-80a9-5e9896241875'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.077194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:46.088812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] The token cache is being invalidate because project fe0488c57a074a8893e4f483b3dd5ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:46.089503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.090831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-9a33af8b-df8c-40d7-80a9-5e9896241875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.092643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 931/1885] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:45 2022] DELETE /identity/v3/projects/fe0488c57a074a8893e4f483b3dd5ae8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:46.098562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.099045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.099366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.124163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:46.125392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:46.205096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 955/1886] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:46.210698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca763419-6cb0-4586-bd48-85952ecbb68a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.212175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca763419-6cb0-4586-bd48-85952ecbb68a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.212676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca763419-6cb0-4586-bd48-85952ecbb68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.231588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.235904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:46.242853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ca763419-6cb0-4586-bd48-85952ecbb68a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:46.245075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ca763419-6cb0-4586-bd48-85952ecbb68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:46.258592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 932/1887] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:46.260162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca763419-6cb0-4586-bd48-85952ecbb68a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.263703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13acf-8664-4c22-8df4-a1e7d875455f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.270319 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.271215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.271494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.271764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] PATH_INFO: `/v3/projects/c95c7335d30a4c4bb27e3f44a54e1792` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.272487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca763419-6cb0-4586-bd48-85952ecbb68a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.273001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95c7335d30a4c4bb27e3f44a54e1792)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.275139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.278152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.278863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.279130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.279401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.280123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:46.280425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.284886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.299201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:46.299561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 933/1888] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:19:46.304035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.320226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94a12e-ce67-4fd6-90a4-31687a6d85ca admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.325788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.326559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.326911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.327262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.328034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.330838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.342730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95c7335d30a4c4bb27e3f44a54e1792', 'request_id': 'req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.359777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 934/1889] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:46 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:46.360437 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95c7335d30a4c4bb27e3f44a54e1792', 'request_id': 'req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.364043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.368045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95c7335d30a4c4bb27e3f44a54e1792', 'request_id': 'req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.380724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] The token cache is being invalidate because project c95c7335d30a4c4bb27e3f44a54e1792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:46.381024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.381704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.383554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 956/1890] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:46 2022] DELETE /identity/v3/projects/c95c7335d30a4c4bb27e3f44a54e1792 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:46.387269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.388681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:46.441959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd0604-0cc4-418a-9a14-63477dc4f481 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.448193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.449614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.450355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.450948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.452033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.455314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.462127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7c8be-1495-45bf-9c85-bdc59f5cf5ab admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.472025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.477278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.478745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.479040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] PATH_INFO: `/v3/users/9394e5cb70774e70862494404db6997d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.480455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9394e5cb70774e70862494404db6997d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.482521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.494060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 935/1891] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:46 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:46.498760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.500659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:46.518882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82651ce5-82b2-4d16-9765-9c0f99188607 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.525698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.526442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.526733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.527010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.527874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.529663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.540946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 936/1892] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:46.544686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.572021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758ff39-9b9e-4542-ba40-0a12a15a6ad9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.572021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.572021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.572021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.572437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.572437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.573738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.578775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9394e5cb70774e70862494404db6997d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.592540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9394e5cb70774e70862494404db6997d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.594588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 937/1893] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:46.599383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.606998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9394e5cb70774e70862494404db6997d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.608446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9394e5cb70774e70862494404db6997d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.609128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9ab6ad14-1ce1-42fa-a453-298009534c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9394e5cb70774e70862494404db6997d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:46.612232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23218aa7-1908-4dd9-acff-10c661425075 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.619795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.621914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.622555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.622851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] PATH_INFO: `/v3/projects/dfcda488969e495193be3062f8995ebf/users/17d97382cb4f437db348e76300b47c71/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.646901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcda488969e495193be3062f8995ebf, user_id=17d97382cb4f437db348e76300b47c71, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.647062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 957/1894] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:19:46 2022] DELETE /identity/v3/users/9394e5cb70774e70862494404db6997d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:46.649612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.670826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 938/1895] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:46 2022] PUT /identity/v3/projects/dfcda488969e495193be3062f8995ebf/users/17d97382cb4f437db348e76300b47c71/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:46.681375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4795-9150-4b90-86e9-1eeb42ae03f7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.681659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4795-9150-4b90-86e9-1eeb42ae03f7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.681938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4795-9150-4b90-86e9-1eeb42ae03f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.711223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1f7c4795-9150-4b90-86e9-1eeb42ae03f7 None None] MFA Rules not processed for user `17d97382cb4f437db348e76300b47c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:46.714075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1f7c4795-9150-4b90-86e9-1eeb42ae03f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:46.728989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:46.799382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 958/1896] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:46.804253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.806225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:46.890870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:46.896083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62e92f-5553-4c6d-b1fc-bfb6c5cb34e4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:46.902955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:46.903696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:46.904045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:46.904737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:46.905392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:46.907397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:46.936416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 939/1897] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:46 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:46.940948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4795-9150-4b90-86e9-1eeb42ae03f7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:46.952131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4795-9150-4b90-86e9-1eeb42ae03f7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.012130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.012865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.013256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.013766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.015067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.018274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.042566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 959/1898] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:46 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:47.046982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.058330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeac60c-c3ae-4a1b-9710-be3b238d098c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.063407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.064621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.064903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.065208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.065987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.067611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.078839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 940/1899] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:47.083345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.095918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a311c-4b30-4eea-822e-e79caf2c0510 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.102041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.103188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.103731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.108369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.109258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.111925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.123343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 960/1900] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:47.129051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.130949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.131069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:47.143282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5d29f-c228-4627-955f-a5f00b0f4966 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.144939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea3610-78bb-4aaf-9fdd-7f9d86eeef99 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.149504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.149735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02462400-bbee-4471-a070-7efd38c996bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.150378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.150654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.150925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] PATH_INFO: `/v3/projects/a66ce97b1caf4766b4804a370d881244` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.151019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462400-bbee-4471-a070-7efd38c996bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.151369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462400-bbee-4471-a070-7efd38c996bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.151735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462400-bbee-4471-a070-7efd38c996bd admin admin] PATH_INFO: `/v3/projects/4d6eae0d25bc4008acc89360e30e6f4f/users/38f1f727c5374230a9927daaae5ff047/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.152236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66ce97b1caf4766b4804a370d881244)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.154158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.169255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02462400-bbee-4471-a070-7efd38c996bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6eae0d25bc4008acc89360e30e6f4f, user_id=38f1f727c5374230a9927daaae5ff047, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.172159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02462400-bbee-4471-a070-7efd38c996bd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.188092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 961/1901] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:47 2022] PUT /identity/v3/projects/4d6eae0d25bc4008acc89360e30e6f4f/users/38f1f727c5374230a9927daaae5ff047/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:47.199339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.199625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.199910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.208821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66ce97b1caf4766b4804a370d881244', 'request_id': 'req-66f2cd37-d1e3-4426-b332-6d0d72c36960'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:47.216290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66ce97b1caf4766b4804a370d881244', 'request_id': 'req-66f2cd37-d1e3-4426-b332-6d0d72c36960'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:47.223274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66ce97b1caf4766b4804a370d881244', 'request_id': 'req-66f2cd37-d1e3-4426-b332-6d0d72c36960'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:47.228925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] MFA Rules not processed for user `38f1f727c5374230a9927daaae5ff047`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:47.230340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:47.235143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] The token cache is being invalidate because project a66ce97b1caf4766b4804a370d881244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:19:47.235374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:47.236066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-66f2cd37-d1e3-4426-b332-6d0d72c36960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:19:47.237704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 941/1902] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:19:47 2022] DELETE /identity/v3/projects/a66ce97b1caf4766b4804a370d881244 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:47.242377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829be2a1-c762-432a-b728-d8cb413d7644 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.242666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829be2a1-c762-432a-b728-d8cb413d7644 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.242951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829be2a1-c762-432a-b728-d8cb413d7644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.260761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-829be2a1-c762-432a-b728-d8cb413d7644 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:47.261973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-829be2a1-c762-432a-b728-d8cb413d7644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:47.293754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:47.301371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 962/1903] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:47.302258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.304008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:47.323890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 942/1904] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:47.332252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26008-e262-41b3-9fc5-b31df6f7e695 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.332541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26008-e262-41b3-9fc5-b31df6f7e695 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.332847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26008-e262-41b3-9fc5-b31df6f7e695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.351178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-26c26008-e262-41b3-9fc5-b31df6f7e695 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:47.352381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-26c26008-e262-41b3-9fc5-b31df6f7e695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:47.361518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 943/1905] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:19:47.365618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26008-e262-41b3-9fc5-b31df6f7e695 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.369093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a1ff4-c737-48e9-a055-cd792fc6417d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.373916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.374581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.374767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26008-e262-41b3-9fc5-b31df6f7e695 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.374864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.375133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.375828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.378081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.379437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.380099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.380387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.380692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.381429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:19:47.381740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.383338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.395898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 944/1906] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:19:47.401026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.413900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 963/1907] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:47.414092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f3af8-71e1-4bc5-abe0-23c986ae6de0 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.417741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.457846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:47.486011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.486591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.486869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.487142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.487832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.489782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.498851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05bc5e-06d5-4a98-b759-261584f24090 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.504369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.505225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.505519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.505799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.506497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.508431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.527054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 945/1908] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:47.533796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.534331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 964/1909] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:47.535208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.551966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c2b3a-08aa-448b-b5a9-42a912f030dc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.599230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e252063-5bd7-417f-a286-4d10732468d5 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.604932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.605634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.605943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.606220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.606915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.608865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.618248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.618919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.619201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.619478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.620209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.621927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.632135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 946/1910] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:47.632951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 965/1911] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:47.636762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.638424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.647537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214d25e-0030-4913-aebc-d8332bd88ebb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.650081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a7f5d-3f3e-4411-b8a0-8e86d3ef0ed3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.652790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.653550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.653839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.654118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.655518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.656461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.658369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.658828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.658983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.659257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] PATH_INFO: `/v3/projects/09368f8e531d4ae0b9a95c9edf3b610c/users/c8aab1ea0ee04839966e02f3fa332161/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.668791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:47.677798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=09368f8e531d4ae0b9a95c9edf3b610c, user_id=c8aab1ea0ee04839966e02f3fa332161, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.680314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 947/1912] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:47.681585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.685103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.699839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a2a3c-d812-476c-ab86-4a7a456374cf admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.703061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 966/1913] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:47 2022] PUT /identity/v3/projects/09368f8e531d4ae0b9a95c9edf3b610c/users/c8aab1ea0ee04839966e02f3fa332161/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:47.704317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.779543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.780366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.780744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.781096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.781897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.784121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.791177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b756c5-f627-448e-87f6-57e66612e0ad admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.795541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 948/1914] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:47.807048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.807624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.807742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.808026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.808307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.809150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.810860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.820169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb7e5-4907-4afd-bfec-0f3077883dbb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.821801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 967/1915] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:47.825530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:47.829247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1eca6f-e0f5-4267-a079-9a900d2a37f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.830495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eca6f-e0f5-4267-a079-9a900d2a37f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.831008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eca6f-e0f5-4267-a079-9a900d2a37f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.831424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eca6f-e0f5-4267-a079-9a900d2a37f6 admin admin] PATH_INFO: `/v3/projects/81a92681ac9d4ad88cc9787123896365/users/8b1a56b5abfc4954a84ab3f1caea7930/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.836332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e611a1e-821c-4800-98e9-ca4aba7cb019 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:47.841154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f9da-b83c-42bd-9a3e-d5d517f59db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:47.842060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f9da-b83c-42bd-9a3e-d5d517f59db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.842342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f9da-b83c-42bd-9a3e-d5d517f59db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.842626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f9da-b83c-42bd-9a3e-d5d517f59db2 admin admin] PATH_INFO: `/v3/projects/09368f8e531d4ae0b9a95c9edf3b610c/users/c8aab1ea0ee04839966e02f3fa332161/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.844665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7f9da-b83c-42bd-9a3e-d5d517f59db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09368f8e531d4ae0b9a95c9edf3b610c, user_id=c8aab1ea0ee04839966e02f3fa332161, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.847995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7f9da-b83c-42bd-9a3e-d5d517f59db2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.850740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1eca6f-e0f5-4267-a079-9a900d2a37f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a92681ac9d4ad88cc9787123896365, user_id=8b1a56b5abfc4954a84ab3f1caea7930, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:47.853482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1eca6f-e0f5-4267-a079-9a900d2a37f6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:47.859068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 968/1916] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:47 2022] PUT /identity/v3/projects/09368f8e531d4ae0b9a95c9edf3b610c/users/c8aab1ea0ee04839966e02f3fa332161/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:47.873307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.873583 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.873886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.889131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 949/1917] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:47 2022] PUT /identity/v3/projects/81a92681ac9d4ad88cc9787123896365/users/8b1a56b5abfc4954a84ab3f1caea7930/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:19:47.900871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b660e-0790-41b9-90dc-0a28b3276fc0 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:47.901293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b660e-0790-41b9-90dc-0a28b3276fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:47.901572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b660e-0790-41b9-90dc-0a28b3276fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:47.902178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] MFA Rules not processed for user `c8aab1ea0ee04839966e02f3fa332161`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:47.903737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:47.922555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:47.934538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-328b660e-0790-41b9-90dc-0a28b3276fc0 None None] MFA Rules not processed for user `8b1a56b5abfc4954a84ab3f1caea7930`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:47.936403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-328b660e-0790-41b9-90dc-0a28b3276fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:47.992206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 969/1918] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:48.004834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.007220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:48.021710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 950/1919] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:48.028089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b660e-0790-41b9-90dc-0a28b3276fc0 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.073914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0afd4-11f1-40a6-b841-0f248dcc49e2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.077991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.078732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.079269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.080170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.082175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:48.092836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:48.105942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b660e-0790-41b9-90dc-0a28b3276fc0 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.111915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.112922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.112922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.113161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.113859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.115872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.164403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 951/1920] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:48 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:48.165499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.167842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:48.182190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.183915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.196308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 970/1921] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:19:48.197366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.212595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb948b-4fdd-4c4e-ab68-bb0c915f0ea2 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.241847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfe593-c9ff-4a50-b732-0aba766bab2f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.245864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.246462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.246747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.247024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.249548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:48.289492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.290287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.290666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.291020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.291906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.294621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.305044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:48.337085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.337085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.339179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 971/1922] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:48 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:48.340275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.354563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9649-6621-40df-abdf-af0e434130a7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.357858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 952/1923] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:48.375682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.378017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.378891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.379754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.383137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.385514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.412325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 972/1924] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:48.413726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.428451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c6bec-c777-4008-8679-3c9dbf84a87e service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.435144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f58e9a-fdc3-4718-8981-91a06ce5a122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.436181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f58e9a-fdc3-4718-8981-91a06ce5a122 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.436457 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f58e9a-fdc3-4718-8981-91a06ce5a122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.436886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f58e9a-fdc3-4718-8981-91a06ce5a122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.437719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f58e9a-fdc3-4718-8981-91a06ce5a122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.440273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f58e9a-fdc3-4718-8981-91a06ce5a122 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.458434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 953/1925] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:19:48.462397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.475949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a86430-be11-47cc-9359-d83dd4d6bbbe admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.486022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.487120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.488705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.488934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] PATH_INFO: `/v3/projects/484168f84ed7401ba160cd1e4ed26842/users/7d7acc7372bf48f9866f3d5454f86a5c/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.512548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=484168f84ed7401ba160cd1e4ed26842, user_id=7d7acc7372bf48f9866f3d5454f86a5c, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.516821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.537709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 973/1926] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:48 2022] PUT /identity/v3/projects/484168f84ed7401ba160cd1e4ed26842/users/7d7acc7372bf48f9866f3d5454f86a5c/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:48.548597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758bfa4-47f3-4ede-a3a1-fa7308dd88c9 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.548929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758bfa4-47f3-4ede-a3a1-fa7308dd88c9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.549208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758bfa4-47f3-4ede-a3a1-fa7308dd88c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.609265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7758bfa4-47f3-4ede-a3a1-fa7308dd88c9 None None] MFA Rules not processed for user `7d7acc7372bf48f9866f3d5454f86a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:48.613122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7758bfa4-47f3-4ede-a3a1-fa7308dd88c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:48.788090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 954/1927] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:48.793651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.884279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88943249-952b-4caf-8a02-947127904eaa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:48.891410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:48.892750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:48.893193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:48.893539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:48.894265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:48.896206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:48.906344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:48.931864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 974/1928] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:19:48 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:19:48.936316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758bfa4-47f3-4ede-a3a1-fa7308dd88c9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:48.958417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758bfa4-47f3-4ede-a3a1-fa7308dd88c9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:49.047731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:49.049200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.049492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.049985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:49.050682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:49.056786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:49.080978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 955/1929] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:19:48 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:19:49.084801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:49.096677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3d928-fb45-49c0-98da-c5fe3769ac4c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:49.102141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:49.103321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.103888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.104370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:49.105521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:49.107205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:49.127263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 975/1930] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:49.129678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:49.140436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe48bc-08ee-414e-bc01-8cc773b308b8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:49.145995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:49.147051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.147338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.147616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] PATH_INFO: `/v3/projects/dc714085beda4d2980fd415c58cb256c/users/fc59fc4ef83d4c9997a9b9aa7f20911e/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:49.165033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc714085beda4d2980fd415c58cb256c, user_id=fc59fc4ef83d4c9997a9b9aa7f20911e, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:49.167939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:49.184925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 956/1931] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:49 2022] PUT /identity/v3/projects/dc714085beda4d2980fd415c58cb256c/users/fc59fc4ef83d4c9997a9b9aa7f20911e/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:49.188941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:49.199950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b7943-e416-4e41-a52f-d85718b136f8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:49.265840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e17b6-24bc-4a12-a593-6a0d036b0b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:49.266526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e17b6-24bc-4a12-a593-6a0d036b0b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.266816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e17b6-24bc-4a12-a593-6a0d036b0b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.267113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e17b6-24bc-4a12-a593-6a0d036b0b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:49.267849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e17b6-24bc-4a12-a593-6a0d036b0b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:49.269514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e17b6-24bc-4a12-a593-6a0d036b0b09 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:49.279634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 976/1932] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:19:49.284584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:49.295187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c3b0-ffa3-4d41-9f6b-28857388bb10 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:49.300529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:49.301461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.301746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.302029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] PATH_INFO: `/v3/projects/dc714085beda4d2980fd415c58cb256c/users/fc59fc4ef83d4c9997a9b9aa7f20911e/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:49.304314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc714085beda4d2980fd415c58cb256c, user_id=fc59fc4ef83d4c9997a9b9aa7f20911e, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:49.307019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:49.323155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 957/1933] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:19:49 2022] PUT /identity/v3/projects/dc714085beda4d2980fd415c58cb256c/users/fc59fc4ef83d4c9997a9b9aa7f20911e/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:19:49.334493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38252-7f05-4aa9-81a9-b019b199af2c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.334780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38252-7f05-4aa9-81a9-b019b199af2c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.335057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38252-7f05-4aa9-81a9-b019b199af2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:49.363397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-19f38252-7f05-4aa9-81a9-b019b199af2c None None] MFA Rules not processed for user `fc59fc4ef83d4c9997a9b9aa7f20911e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:49.365010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-19f38252-7f05-4aa9-81a9-b019b199af2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:49.437295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 977/1934] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:49.456173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:49.467407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:49.517820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4c12-266f-46cb-983d-d9894454c678 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:49.522305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:49.522981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.523416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.523703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:49.534532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:49.536341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:49.551757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 958/1935] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:49.984241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38252-7f05-4aa9-81a9-b019b199af2c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:49.994646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38252-7f05-4aa9-81a9-b019b199af2c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:49.998433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:49.999085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:49.999368 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:49.999646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:50.064469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:50.065972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:50.069776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:19:50.076659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 978/1936] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:50.170519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:50.179668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f98e-26e7-47bf-95fe-0a3ccce0958a service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:50.183701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c07243-3e31-45ff-a0bc-cc62bf7b2c6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:50.184345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07243-3e31-45ff-a0bc-cc62bf7b2c6e service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:50.186290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07243-3e31-45ff-a0bc-cc62bf7b2c6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:50.186733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c07243-3e31-45ff-a0bc-cc62bf7b2c6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:50.266576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c07243-3e31-45ff-a0bc-cc62bf7b2c6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:50.268208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c07243-3e31-45ff-a0bc-cc62bf7b2c6e service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:50.278550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 959/1937] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:51.603125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:51.613685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12967897-ddf3-4220-aa4d-44b316aa96c5 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:51.617463 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcc7dc-b591-40dd-8dff-f180e8c8f623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:51.618191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc7dc-b591-40dd-8dff-f180e8c8f623 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:51.618540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc7dc-b591-40dd-8dff-f180e8c8f623 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:51.618889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc7dc-b591-40dd-8dff-f180e8c8f623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:51.685549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc7dc-b591-40dd-8dff-f180e8c8f623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:51.686991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc7dc-b591-40dd-8dff-f180e8c8f623 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:51.696718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 979/1938] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:19:54.843748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5408a-0e93-42cb-b4ce-892ae473e1c2 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:54.844309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5408a-0e93-42cb-b4ce-892ae473e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:54.844309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5408a-0e93-42cb-b4ce-892ae473e1c2 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:54.846563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 960/1939] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:19:54.855177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebed0a-a708-4d94-8924-3f761865da29 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:54.855701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebed0a-a708-4d94-8924-3f761865da29 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:54.855778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebed0a-a708-4d94-8924-3f761865da29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:54.874742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9eebed0a-a708-4d94-8924-3f761865da29 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:19:54.877394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9eebed0a-a708-4d94-8924-3f761865da29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:54.977371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 980/1940] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:19:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:19:55.002575 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5408a-0e93-42cb-b4ce-892ae473e1c2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:19:55.005099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0fa5408a-0e93-42cb-b4ce-892ae473e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:19:55.024507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5408a-0e93-42cb-b4ce-892ae473e1c2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:19:55.030733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:19:55.033344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:19:55.033344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:19:55.033344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:19:55.042576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:19:55.044186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:19:55.055929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 961/1941] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:19:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:03.777811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebed0a-a708-4d94-8924-3f761865da29 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:03.787691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebed0a-a708-4d94-8924-3f761865da29 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:03.791407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcad05-d246-4274-914d-81a98e943d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:03.792135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcad05-d246-4274-914d-81a98e943d47 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:03.792416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcad05-d246-4274-914d-81a98e943d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:03.792723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcad05-d246-4274-914d-81a98e943d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:03.848375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcad05-d246-4274-914d-81a98e943d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:03.849756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcad05-d246-4274-914d-81a98e943d47 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:03.858950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 981/1942] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:04.306499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:04.389894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d683e-7127-42d6-9a43-839cd227f024 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:04.397000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:04.398519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:04.399018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:04.399462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] PATH_INFO: `/v3/users/17d97382cb4f437db348e76300b47c71` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:04.401023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d97382cb4f437db348e76300b47c71)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:04.403342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:04.471651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:04.484300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d97382cb4f437db348e76300b47c71'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:04.498377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d97382cb4f437db348e76300b47c71'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:04.507586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d97382cb4f437db348e76300b47c71'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:04.508289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d97382cb4f437db348e76300b47c71'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:04.512994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d97382cb4f437db348e76300b47c71'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:04.549091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 962/1943] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:04 2022] DELETE /identity/v3/users/17d97382cb4f437db348e76300b47c71 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:20:04.554550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:04.554945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:04.555324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:04.580324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:04.581924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:04.612938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:04.678629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 982/1944] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:04.690490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:04.692280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:04.765356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ab7d7-b97f-471b-ab67-2924d7fc32ef admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:04.769043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:04.770350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:04.770633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:04.770909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:04.782344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:04.784203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:04.795110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 963/1945] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:05.216848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:05.218333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:05.231427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99675241-0e22-48ff-be79-2bfb27794205 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:05.236455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:05.237393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:05.237675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:05.237950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] PATH_INFO: `/v3/projects/dfcda488969e495193be3062f8995ebf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:05.239168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfcda488969e495193be3062f8995ebf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:05.241169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:05.308466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfcda488969e495193be3062f8995ebf', 'request_id': 'req-fef03e29-249d-4603-8196-44a4d4ef8fe0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.316468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfcda488969e495193be3062f8995ebf', 'request_id': 'req-fef03e29-249d-4603-8196-44a4d4ef8fe0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.322833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfcda488969e495193be3062f8995ebf', 'request_id': 'req-fef03e29-249d-4603-8196-44a4d4ef8fe0'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.334994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] The token cache is being invalidate because project dfcda488969e495193be3062f8995ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:05.335434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.335994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.337625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 983/1946] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:05 2022] DELETE /identity/v3/projects/dfcda488969e495193be3062f8995ebf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:05.341679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:05.352682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05537a06-0df5-4701-90e6-5bc648f2ef8c service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:05.418676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:05.419629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:05.420087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:05.420513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] PATH_INFO: `/v3/users/38f1f727c5374230a9927daaae5ff047` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:05.422440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f1f727c5374230a9927daaae5ff047)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:05.425054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:05.492477 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f1f727c5374230a9927daaae5ff047'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.500414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f1f727c5374230a9927daaae5ff047'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.505521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:05.509394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f1f727c5374230a9927daaae5ff047'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.510238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f1f727c5374230a9927daaae5ff047'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.511410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f1f727c5374230a9927daaae5ff047'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.531135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 964/1947] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:05 2022] DELETE /identity/v3/users/38f1f727c5374230a9927daaae5ff047 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:20:05.823614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:05.825651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:05.839748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef03e29-249d-4603-8196-44a4d4ef8fe0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:05.911577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:05.912499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:05.912845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:05.913294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] PATH_INFO: `/v3/projects/4d6eae0d25bc4008acc89360e30e6f4f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:05.914638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6eae0d25bc4008acc89360e30e6f4f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:05.916672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:05.964886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:05.966129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6eae0d25bc4008acc89360e30e6f4f', 'request_id': 'req-a4583548-5cd4-4a66-8086-7e6e7092e656'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.973870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6eae0d25bc4008acc89360e30e6f4f', 'request_id': 'req-a4583548-5cd4-4a66-8086-7e6e7092e656'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.979797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6eae0d25bc4008acc89360e30e6f4f', 'request_id': 'req-a4583548-5cd4-4a66-8086-7e6e7092e656'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.991425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] The token cache is being invalidate because project 4d6eae0d25bc4008acc89360e30e6f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:05.991714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.992356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:05.993956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 984/1948] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:05 2022] DELETE /identity/v3/projects/4d6eae0d25bc4008acc89360e30e6f4f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:06.003369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:06.004786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:06.017833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbacc24-4e18-4a4d-944c-fd87d7f515bb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:06.086950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:06.087995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:06.088340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:06.089254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] PATH_INFO: `/v3/users/c8aab1ea0ee04839966e02f3fa332161` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:06.090478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8aab1ea0ee04839966e02f3fa332161)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:06.092431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:06.145692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8aab1ea0ee04839966e02f3fa332161'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.154723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8aab1ea0ee04839966e02f3fa332161'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.163117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8aab1ea0ee04839966e02f3fa332161'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.163886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8aab1ea0ee04839966e02f3fa332161'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.164526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-14b638d8-cd0c-4100-b54c-5f9078084232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8aab1ea0ee04839966e02f3fa332161'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.189819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 965/1949] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:06 2022] DELETE /identity/v3/users/c8aab1ea0ee04839966e02f3fa332161 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:06.470218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:06.472183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:06.491465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4583548-5cd4-4a66-8086-7e6e7092e656 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:06.550384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:06.551342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:06.551713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:06.552056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] PATH_INFO: `/v3/projects/09368f8e531d4ae0b9a95c9edf3b610c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:06.553346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09368f8e531d4ae0b9a95c9edf3b610c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:06.555585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:06.604590 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09368f8e531d4ae0b9a95c9edf3b610c', 'request_id': 'req-1f0e5566-159e-415b-b9fe-de42c5150e02'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.614059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09368f8e531d4ae0b9a95c9edf3b610c', 'request_id': 'req-1f0e5566-159e-415b-b9fe-de42c5150e02'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.620565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09368f8e531d4ae0b9a95c9edf3b610c', 'request_id': 'req-1f0e5566-159e-415b-b9fe-de42c5150e02'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.634565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] The token cache is being invalidate because project 09368f8e531d4ae0b9a95c9edf3b610c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:06.634913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.635582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f0e5566-159e-415b-b9fe-de42c5150e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:06.637373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 985/1950] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:06 2022] DELETE /identity/v3/projects/09368f8e531d4ae0b9a95c9edf3b610c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:06.642123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee387f-f976-4b37-a5a6-dcdb68f0660e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:06.642593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee387f-f976-4b37-a5a6-dcdb68f0660e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:06.642694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee387f-f976-4b37-a5a6-dcdb68f0660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:06.661670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ffee387f-f976-4b37-a5a6-dcdb68f0660e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:06.662876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ffee387f-f976-4b37-a5a6-dcdb68f0660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:06.713395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:06.727460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 966/1951] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:06.736232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2813e-2755-4371-ad9e-d3dc4f50b52a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:06.736516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2813e-2755-4371-ad9e-d3dc4f50b52a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:06.736824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2813e-2755-4371-ad9e-d3dc4f50b52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:06.754786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bfd2813e-2755-4371-ad9e-d3dc4f50b52a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:06.756003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bfd2813e-2755-4371-ad9e-d3dc4f50b52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:06.764139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 986/1952] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:06.768811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee387f-f976-4b37-a5a6-dcdb68f0660e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:06.778358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee387f-f976-4b37-a5a6-dcdb68f0660e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:06.783623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:06.784276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:06.784657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:06.784996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:06.785578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:20:06.785884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:06.787466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:06.799333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 967/1953] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:20:06.804425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2813e-2755-4371-ad9e-d3dc4f50b52a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:06.813776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2813e-2755-4371-ad9e-d3dc4f50b52a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:06.818600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:06.819253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:06.819535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:06.819813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:06.820508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:06.822437 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:06.853403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 987/1954] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:06 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:20:06.857625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:06.871623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ab1f-efec-4bec-ae9e-01b9c3df4399 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:06.954982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:06.955815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:06.956106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:06.956381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:06.957113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:06.959019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:06.993668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 968/1955] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:06 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:20:06.998075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:07.008888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e0d7-273f-4d3e-a114-66f63b4ef379 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:07.014593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:07.015348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:07.015697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:07.016043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:07.017439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:07.024617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:07.045946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 988/1956] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:07.051816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:07.064340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387fa3f-a9e8-44df-a58b-7b3477300876 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:07.070204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53241c-dd6c-445d-9585-bcf3b081c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:07.070868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53241c-dd6c-445d-9585-bcf3b081c319 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:07.071152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53241c-dd6c-445d-9585-bcf3b081c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:07.071430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53241c-dd6c-445d-9585-bcf3b081c319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:07.072164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53241c-dd6c-445d-9585-bcf3b081c319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:07.073808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53241c-dd6c-445d-9585-bcf3b081c319 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:07.085591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 969/1957] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:20:07.089864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:07.102513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f01fb-b5b1-47d9-bd5e-ce53792e8b53 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:07.110542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:07.112129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:07.112798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:07.113292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] PATH_INFO: `/v3/projects/1381dac7b3a64ee5a3ceeed2f1406a12/users/648868f0f8674fe294df4b0ce2e9bdf6/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:07.132345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1381dac7b3a64ee5a3ceeed2f1406a12, user_id=648868f0f8674fe294df4b0ce2e9bdf6, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:07.135054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:07.149888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 989/1958] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:07 2022] PUT /identity/v3/projects/1381dac7b3a64ee5a3ceeed2f1406a12/users/648868f0f8674fe294df4b0ce2e9bdf6/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:07.160451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:07.160775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:07.161056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:07.193202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] MFA Rules not processed for user `648868f0f8674fe294df4b0ce2e9bdf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:07.194475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:07.277773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 970/1959] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:07.288868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:07.290616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:07.295969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:07.369207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c468-c562-4356-8b89-82f4ee1c9827 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:07.372906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61acd5-5726-4b9a-be96-0c6b831cfcdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:07.373574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61acd5-5726-4b9a-be96-0c6b831cfcdc service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:07.373871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61acd5-5726-4b9a-be96-0c6b831cfcdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:07.374143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61acd5-5726-4b9a-be96-0c6b831cfcdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:07.383745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61acd5-5726-4b9a-be96-0c6b831cfcdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:07.385253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61acd5-5726-4b9a-be96-0c6b831cfcdc service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:07.394833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 990/1960] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:11.727458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:11.729072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:11.813188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:11.861727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0049b91-1219-4637-a49f-09f4c7a08f92 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:11.868360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:11.870615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:11.870907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:11.871346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] PATH_INFO: `/v3/users/8b1a56b5abfc4954a84ab3f1caea7930` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:11.873701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1a56b5abfc4954a84ab3f1caea7930)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:11.876126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:11.992851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1a56b5abfc4954a84ab3f1caea7930'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.021003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1a56b5abfc4954a84ab3f1caea7930'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.045033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1a56b5abfc4954a84ab3f1caea7930'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.046521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1a56b5abfc4954a84ab3f1caea7930'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.049668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1a56b5abfc4954a84ab3f1caea7930'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.096507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 971/1961] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:11 2022] DELETE /identity/v3/users/8b1a56b5abfc4954a84ab3f1caea7930 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:20:12.101056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:12.101595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:12.101595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:12.124502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:12.125884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:12.258379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 991/1962] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:12.281550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:12.286563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:12.378383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836f58-830d-49b4-9f0d-7ef3fd3d1557 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:12.382992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:12.383852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:12.384142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:12.384590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:12.396708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:12.398189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:12.411142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 972/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:12.853914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:12.855369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:12.869156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fdafc-3265-44fa-80bf-4f3f5e73160e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:12.874591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:12.875593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:12.876408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:12.876860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] PATH_INFO: `/v3/projects/81a92681ac9d4ad88cc9787123896365` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:12.879827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a92681ac9d4ad88cc9787123896365)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:12.882327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:12.936019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a92681ac9d4ad88cc9787123896365', 'request_id': 'req-83d1a601-813b-4aa7-934e-948c500f56aa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.944387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a92681ac9d4ad88cc9787123896365', 'request_id': 'req-83d1a601-813b-4aa7-934e-948c500f56aa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.950375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a92681ac9d4ad88cc9787123896365', 'request_id': 'req-83d1a601-813b-4aa7-934e-948c500f56aa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.962786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] The token cache is being invalidate because project 81a92681ac9d4ad88cc9787123896365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:12.963071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.963731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:12.965334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 992/1964] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:12 2022] DELETE /identity/v3/projects/81a92681ac9d4ad88cc9787123896365 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:12.970211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:12.980275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b3883-97c7-446b-ac08-23baa662ac59 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:13.042434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:13.043383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:13.043742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:13.044094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] PATH_INFO: `/v3/users/7d7acc7372bf48f9866f3d5454f86a5c` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:13.046009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d7acc7372bf48f9866f3d5454f86a5c)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:13.048815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:13.066297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:13.123020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d7acc7372bf48f9866f3d5454f86a5c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.130732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d7acc7372bf48f9866f3d5454f86a5c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.138837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d7acc7372bf48f9866f3d5454f86a5c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.139479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d7acc7372bf48f9866f3d5454f86a5c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.140206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d7acc7372bf48f9866f3d5454f86a5c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.160445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 973/1965] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:12 2022] DELETE /identity/v3/users/7d7acc7372bf48f9866f3d5454f86a5c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:13.449150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:13.450639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:13.465505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a601-813b-4aa7-934e-948c500f56aa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:13.496140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:13.532541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:13.533839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:13.534125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:13.534403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] PATH_INFO: `/v3/projects/484168f84ed7401ba160cd1e4ed26842` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:13.535644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=484168f84ed7401ba160cd1e4ed26842)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:13.537614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:13.590364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484168f84ed7401ba160cd1e4ed26842', 'request_id': 'req-02cd989d-6d30-495f-8a84-894e29cfcb2b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.601924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484168f84ed7401ba160cd1e4ed26842', 'request_id': 'req-02cd989d-6d30-495f-8a84-894e29cfcb2b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.609296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484168f84ed7401ba160cd1e4ed26842', 'request_id': 'req-02cd989d-6d30-495f-8a84-894e29cfcb2b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.622713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] The token cache is being invalidate because project 484168f84ed7401ba160cd1e4ed26842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:13.623009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.623692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.625511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 993/1966] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:13 2022] DELETE /identity/v3/projects/484168f84ed7401ba160cd1e4ed26842 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:13.629429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:13.631073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:13.645244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6926e-f8e6-4333-8055-08f0e82e2899 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:13.714649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:13.715562 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:13.715999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:13.716452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] PATH_INFO: `/v3/users/fc59fc4ef83d4c9997a9b9aa7f20911e` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:13.718108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc59fc4ef83d4c9997a9b9aa7f20911e)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:13.722073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:13.786284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc59fc4ef83d4c9997a9b9aa7f20911e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.794379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc59fc4ef83d4c9997a9b9aa7f20911e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.804181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc59fc4ef83d4c9997a9b9aa7f20911e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.805014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc59fc4ef83d4c9997a9b9aa7f20911e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.805683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2baf562f-302f-4d30-9ee9-522dfc0f1099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc59fc4ef83d4c9997a9b9aa7f20911e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:13.833489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 974/1967] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:13 2022] DELETE /identity/v3/users/fc59fc4ef83d4c9997a9b9aa7f20911e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:20:14.209329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:14.210775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:14.227247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd989d-6d30-495f-8a84-894e29cfcb2b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:14.292790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:14.293803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.294209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.294209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] PATH_INFO: `/v3/projects/dc714085beda4d2980fd415c58cb256c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.295726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc714085beda4d2980fd415c58cb256c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:14.298234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:14.345160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc714085beda4d2980fd415c58cb256c', 'request_id': 'req-7bd237c4-88dc-46ee-a022-4b69e04f3c67'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:14.352417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc714085beda4d2980fd415c58cb256c', 'request_id': 'req-7bd237c4-88dc-46ee-a022-4b69e04f3c67'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:14.356279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:14.358639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc714085beda4d2980fd415c58cb256c', 'request_id': 'req-7bd237c4-88dc-46ee-a022-4b69e04f3c67'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:14.371572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] The token cache is being invalidate because project dc714085beda4d2980fd415c58cb256c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:14.371879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:14.372510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7bd237c4-88dc-46ee-a022-4b69e04f3c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:14.374069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 994/1968] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:14 2022] DELETE /identity/v3/projects/dc714085beda4d2980fd415c58cb256c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:14.380467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f21ca-debc-4f2e-8dfd-4305807f55fa None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.380970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f21ca-debc-4f2e-8dfd-4305807f55fa None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.381060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f21ca-debc-4f2e-8dfd-4305807f55fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.399301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-cd8f21ca-debc-4f2e-8dfd-4305807f55fa None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:14.400579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cd8f21ca-debc-4f2e-8dfd-4305807f55fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:14.406076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:14.463113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 975/1969] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:14.471504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e37d3e-96ef-49e6-8b66-81a324921da5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.471998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e37d3e-96ef-49e6-8b66-81a324921da5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.472132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e37d3e-96ef-49e6-8b66-81a324921da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.490667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-59e37d3e-96ef-49e6-8b66-81a324921da5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:14.491874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-59e37d3e-96ef-49e6-8b66-81a324921da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:14.499808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 995/1970] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:14.504687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f21ca-debc-4f2e-8dfd-4305807f55fa None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:14.514552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f21ca-debc-4f2e-8dfd-4305807f55fa None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:14.519567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:14.520363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.520689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.520969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.521697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:20:14.522005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:14.523628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:14.536368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 976/1971] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:20:14.540457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e37d3e-96ef-49e6-8b66-81a324921da5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:14.550622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e37d3e-96ef-49e6-8b66-81a324921da5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:14.556331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:14.557153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.557523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.557880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.558706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:14.560878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:14.589401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 996/1972] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:14 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:14.593696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:14.607372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650d651-a6ae-4b72-b52e-ff2d2376fdc6 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:14.669311 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:14.669994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.670279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.670556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.671254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:14.673809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:14.701941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 977/1973] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:14 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:14.703192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:14.714030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da939-7843-4ebf-80e2-d14ef64221a7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:14.719008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:14.720220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.720502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.720807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.721541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:14.723133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:14.738299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 997/1974] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:20:14.741779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:14.752309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3c103-f68e-45b1-8609-683b3cc0b048 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:14.757675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62342e-06b8-480f-85d3-6efca8895209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:14.758319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62342e-06b8-480f-85d3-6efca8895209 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.758662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62342e-06b8-480f-85d3-6efca8895209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.759026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62342e-06b8-480f-85d3-6efca8895209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.760333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62342e-06b8-480f-85d3-6efca8895209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:14.761965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62342e-06b8-480f-85d3-6efca8895209 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:14.779750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 978/1975] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:14.779972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:14.792192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0db1e-5671-4388-bfc0-aadf23d3fc7a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:14.801716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:14.802617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.802898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.803173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] PATH_INFO: `/v3/projects/c9751f9fcd534ad894bf7c65aba9c272/users/e2ded8edde0c4636a2f92b0d63bfe04f/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.820060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9751f9fcd534ad894bf7c65aba9c272, user_id=e2ded8edde0c4636a2f92b0d63bfe04f, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:14.822740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:14.838323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 998/1976] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:14 2022] PUT /identity/v3/projects/c9751f9fcd534ad894bf7c65aba9c272/users/e2ded8edde0c4636a2f92b0d63bfe04f/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:14.850516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad74a6a-40eb-4163-9623-86e3db28b2d5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:14.850808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad74a6a-40eb-4163-9623-86e3db28b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:14.851089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad74a6a-40eb-4163-9623-86e3db28b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:14.878665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-cad74a6a-40eb-4163-9623-86e3db28b2d5 None None] MFA Rules not processed for user `e2ded8edde0c4636a2f92b0d63bfe04f`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:14.879967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cad74a6a-40eb-4163-9623-86e3db28b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:14.963211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 979/1977] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:14.967718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.045788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2b0d-b643-4d4e-94a7-0e1bd937854d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.051038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.051620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.051908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.052187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.052925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.055101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.092081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 999/1978] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:14 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:15.093141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad74a6a-40eb-4163-9623-86e3db28b2d5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.104256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad74a6a-40eb-4163-9623-86e3db28b2d5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.107796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:15.166106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.166764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.167091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.168346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.168346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.170035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.190796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 980/1979] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:15 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:20:15.194649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.205547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd18787-7679-4ed1-bf74-626b9df32c46 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.210090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.210742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.211023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.211297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.212016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.213625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.224568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1000/1980] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:15.228693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.238707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b840bf-bd2c-4f8f-a9a3-3496e2c20bd2 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.243415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.244301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.244610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.244887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] PATH_INFO: `/v3/projects/90e12b71d6084b1daaf3ca6df08b8475/users/51ac606352e24e16aa18a0b143d1ab79/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.261527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e12b71d6084b1daaf3ca6df08b8475, user_id=51ac606352e24e16aa18a0b143d1ab79, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.264309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.277517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 981/1981] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:15 2022] PUT /identity/v3/projects/90e12b71d6084b1daaf3ca6df08b8475/users/51ac606352e24e16aa18a0b143d1ab79/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:15.281480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.291867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf4726-316a-4589-861c-977edec608d3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.347936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc195bdd-4482-4c41-abfc-89e55c4dba4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.348621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc195bdd-4482-4c41-abfc-89e55c4dba4a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.348906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc195bdd-4482-4c41-abfc-89e55c4dba4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.349208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc195bdd-4482-4c41-abfc-89e55c4dba4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.349922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc195bdd-4482-4c41-abfc-89e55c4dba4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.351532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc195bdd-4482-4c41-abfc-89e55c4dba4a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.352815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:15.362841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1001/1982] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:15.366970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.377712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662b713-f47a-4c00-a499-2dcfb8948ac8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.382282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.383156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.383436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.383738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] PATH_INFO: `/v3/projects/90e12b71d6084b1daaf3ca6df08b8475/users/51ac606352e24e16aa18a0b143d1ab79/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.385720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e12b71d6084b1daaf3ca6df08b8475, user_id=51ac606352e24e16aa18a0b143d1ab79, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.388313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.399255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 982/1983] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:15 2022] PUT /identity/v3/projects/90e12b71d6084b1daaf3ca6df08b8475/users/51ac606352e24e16aa18a0b143d1ab79/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:15.409976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2512-9a1d-4360-bd1f-53279ed2505a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.410265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2512-9a1d-4360-bd1f-53279ed2505a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.410542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2512-9a1d-4360-bd1f-53279ed2505a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.437179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ed5c2512-9a1d-4360-bd1f-53279ed2505a None None] MFA Rules not processed for user `51ac606352e24e16aa18a0b143d1ab79`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:15.438837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ed5c2512-9a1d-4360-bd1f-53279ed2505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:15.505647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1002/1984] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:15.517052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.518749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:15.574951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c96cf-ad55-4a58-82ee-81507e4c5103 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.578691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.579843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.580126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.580406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.589596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.591026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.600741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 983/1985] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:15.699842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c2512-9a1d-4360-bd1f-53279ed2505a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:15.710179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c2512-9a1d-4360-bd1f-53279ed2505a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:15.713741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701806aa-8d76-443f-a2e0-da7305e9f4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:15.714391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701806aa-8d76-443f-a2e0-da7305e9f4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:15.714670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701806aa-8d76-443f-a2e0-da7305e9f4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:15.714946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701806aa-8d76-443f-a2e0-da7305e9f4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:15.776146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701806aa-8d76-443f-a2e0-da7305e9f4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:15.777613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701806aa-8d76-443f-a2e0-da7305e9f4c2 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:15.786784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1003/1986] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:19.491108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:19.557230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5045-b122-4682-96f9-31e3592c5ab3 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:19.562005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:19.563698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:19.563983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:19.564260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] PATH_INFO: `/v3/users/e2ded8edde0c4636a2f92b0d63bfe04f` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:19.565506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ded8edde0c4636a2f92b0d63bfe04f)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:19.567417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:19.570134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:19.626909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ded8edde0c4636a2f92b0d63bfe04f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:19.638998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ded8edde0c4636a2f92b0d63bfe04f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:19.649615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ded8edde0c4636a2f92b0d63bfe04f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:19.650467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ded8edde0c4636a2f92b0d63bfe04f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:19.651232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ded8edde0c4636a2f92b0d63bfe04f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:19.673931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 984/1987] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:19 2022] DELETE /identity/v3/users/e2ded8edde0c4636a2f92b0d63bfe04f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:19.678604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:19.679343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:19.679433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:19.697178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:19.698627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:19.781054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1004/1988] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:19.793072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:19.794808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:19.872812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4dc73-7e8e-451d-99a2-e20be5020284 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:19.876750 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:19.877624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:19.877917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:19.878238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:19.905513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:19.907535 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:19.923729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 985/1989] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:20.230731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:20.232343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:20.252188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172118b9-a207-4ea4-8ca5-11745379fb8a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:20.258504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:20.259408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:20.259716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:20.259961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] PATH_INFO: `/v3/projects/c9751f9fcd534ad894bf7c65aba9c272` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:20.261557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9751f9fcd534ad894bf7c65aba9c272)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:20.263992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:20.287701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:20.332075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9751f9fcd534ad894bf7c65aba9c272', 'request_id': 'req-c2339fcd-5be5-42af-b98e-cd0365954a6f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.348487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9751f9fcd534ad894bf7c65aba9c272', 'request_id': 'req-c2339fcd-5be5-42af-b98e-cd0365954a6f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.357500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9751f9fcd534ad894bf7c65aba9c272', 'request_id': 'req-c2339fcd-5be5-42af-b98e-cd0365954a6f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.398455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] The token cache is being invalidate because project c9751f9fcd534ad894bf7c65aba9c272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:20.398751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.399432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.401604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1005/1990] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:20 2022] DELETE /identity/v3/projects/c9751f9fcd534ad894bf7c65aba9c272 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:20.405563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:20.417403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828d2ea-65f9-4aa2-a757-f5a590b12878 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:20.511411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:20.513870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:20.513870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:20.513870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] PATH_INFO: `/v3/users/51ac606352e24e16aa18a0b143d1ab79` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:20.514557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ac606352e24e16aa18a0b143d1ab79)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:20.517192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:20.554984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:20.575220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ac606352e24e16aa18a0b143d1ab79'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.583805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ac606352e24e16aa18a0b143d1ab79'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.591390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ac606352e24e16aa18a0b143d1ab79'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.592441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ac606352e24e16aa18a0b143d1ab79'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.593206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ac606352e24e16aa18a0b143d1ab79'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:20.611637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 986/1991] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:20 2022] DELETE /identity/v3/users/51ac606352e24e16aa18a0b143d1ab79 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:21.110066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.113274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:21.181744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339fcd-5be5-42af-b98e-cd0365954a6f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.193371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.195452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:21.300711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.301611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.302215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.302215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] PATH_INFO: `/v3/projects/90e12b71d6084b1daaf3ca6df08b8475` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.303450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e12b71d6084b1daaf3ca6df08b8475)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.306111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0304618-8d9c-4eca-ac30-53a5796a50af admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.306278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.311354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.312326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.313414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.314169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] PATH_INFO: `/v3/users/9e78d07a02c34abcafa7ecd57a8ff1d2` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.315466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e78d07a02c34abcafa7ecd57a8ff1d2)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.317424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.361720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e12b71d6084b1daaf3ca6df08b8475', 'request_id': 'req-47633280-6754-4743-9aab-d6651f48420e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.370925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e12b71d6084b1daaf3ca6df08b8475', 'request_id': 'req-47633280-6754-4743-9aab-d6651f48420e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.374766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e78d07a02c34abcafa7ecd57a8ff1d2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.380196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e12b71d6084b1daaf3ca6df08b8475', 'request_id': 'req-47633280-6754-4743-9aab-d6651f48420e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.385812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e78d07a02c34abcafa7ecd57a8ff1d2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.392193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] The token cache is being invalidate because project 90e12b71d6084b1daaf3ca6df08b8475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:21.392484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.393181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-47633280-6754-4743-9aab-d6651f48420e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.394828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e78d07a02c34abcafa7ecd57a8ff1d2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.395010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1006/1992] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:21 2022] DELETE /identity/v3/projects/90e12b71d6084b1daaf3ca6df08b8475 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:21.397171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e78d07a02c34abcafa7ecd57a8ff1d2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.397806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-76bde8f0-6a49-483c-8574-61ffb3333a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e78d07a02c34abcafa7ecd57a8ff1d2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:21.400303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938af9-f144-458b-baff-34b78ddcf8f5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.400623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938af9-f144-458b-baff-34b78ddcf8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.400902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d938af9-f144-458b-baff-34b78ddcf8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.420540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2d938af9-f144-458b-baff-34b78ddcf8f5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:21.421965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 987/1993] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:21 2022] DELETE /identity/v3/users/9e78d07a02c34abcafa7ecd57a8ff1d2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:21.422659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2d938af9-f144-458b-baff-34b78ddcf8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:21.426801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.427444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.427444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.446703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:21.448019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:21.457656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:21.468063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:21.510452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1007/1994] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:21.518962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bb67e-402b-4abc-9181-580556722101 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.518962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bb67e-402b-4abc-9181-580556722101 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.519563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bb67e-402b-4abc-9181-580556722101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.543760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 988/1995] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:20:21.548078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-759bb67e-402b-4abc-9181-580556722101 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:21.549781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-759bb67e-402b-4abc-9181-580556722101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:21.552550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.563966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:21.586345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1008/1996] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:20:21.590428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bb67e-402b-4abc-9181-580556722101 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.603710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bb67e-402b-4abc-9181-580556722101 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.623671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.624351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.624671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.624950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.625722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:20:21.626027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.627633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.644094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1009/1997] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:20:21.644901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8414ac5-351c-43d8-8598-36c6a01c9f1d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.645142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.649132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.649788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.650070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.650345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.657615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37412e14-fcc9-463d-95ed-530781aad33f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.665442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.665442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.665442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.665442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.666058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.668053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.669247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.670633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.689050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 989/1998] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:20:21.715410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1010/1999] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:21 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:21.718231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.732587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbfbdb-dd14-4c44-81cb-18801e2c48a5 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.817648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.819040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.819518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.820798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.821701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.824650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.854185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 990/2000] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:21 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:21.858460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.905311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a5417-d9cd-4dbf-bfe7-86bdbfe06551 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.916519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.918466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.918766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.919046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.919800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.921450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.945761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.961334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1011/2001] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:21.964890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82aef1-8a89-4027-8cc5-0ecbb5acf19d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:21.972768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d55f-f124-485e-9130-670e5962bfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:21.973455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d55f-f124-485e-9130-670e5962bfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:21.973744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d55f-f124-485e-9130-670e5962bfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:21.974020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d55f-f124-485e-9130-670e5962bfb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:21.974756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6d55f-f124-485e-9130-670e5962bfb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:21.976645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6d55f-f124-485e-9130-670e5962bfb4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:21.994879 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:21.996256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 991/2002] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:22.009017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d825a-2228-4edb-b460-03c3ba90b334 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:22.015264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:22.017435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.017727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.018007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] PATH_INFO: `/v3/projects/6b3bcc7e2a2a4ca8a0d6a0e71c393723/users/6267c78d0e3c48b2aed6c4d7ea3cc8ad/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.044020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3bcc7e2a2a4ca8a0d6a0e71c393723, user_id=6267c78d0e3c48b2aed6c4d7ea3cc8ad, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:22.047497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:22.102098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1012/2003] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:21 2022] PUT /identity/v3/projects/6b3bcc7e2a2a4ca8a0d6a0e71c393723/users/6267c78d0e3c48b2aed6c4d7ea3cc8ad/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:22.110973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.111265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.111545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.150727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] MFA Rules not processed for user `6267c78d0e3c48b2aed6c4d7ea3cc8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:22.152200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:22.208523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:22.274732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 992/2004] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:22.290752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:22.293652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:22.346351 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:22.348418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:22.403154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e90ff-5b8a-4311-a7c1-0fc75b391d93 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:22.419953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:22.422795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.423289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.424900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.437334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:22.439733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:22.454723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1013/2005] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:22.469490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0b4c8-28be-4daf-b3e5-84c18d4b73cc None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:22.474151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:22.474679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:22.475843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.476216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.476578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] PATH_INFO: `/v3/projects/57d971df6ca64a209bbecf001aea602f` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.479639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d971df6ca64a209bbecf001aea602f)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:22.481775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:22.489367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d44779-55fd-4d29-8f8c-38ea4b69c766 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:22.495024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:22.496722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.497484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.497949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] PATH_INFO: `/v3/users/6267c78d0e3c48b2aed6c4d7ea3cc8ad` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.501020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6267c78d0e3c48b2aed6c4d7ea3cc8ad)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:22.503147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:22.555305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d971df6ca64a209bbecf001aea602f', 'request_id': 'req-eb066705-05a1-4b8a-819c-5e23462e29a0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.569811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d971df6ca64a209bbecf001aea602f', 'request_id': 'req-eb066705-05a1-4b8a-819c-5e23462e29a0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.579018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6267c78d0e3c48b2aed6c4d7ea3cc8ad'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.580706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d971df6ca64a209bbecf001aea602f', 'request_id': 'req-eb066705-05a1-4b8a-819c-5e23462e29a0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.593072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6267c78d0e3c48b2aed6c4d7ea3cc8ad'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.597744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:22.598001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] The token cache is being invalidate because project 57d971df6ca64a209bbecf001aea602f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:22.598290 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.601544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.607892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 993/2006] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:22 2022] DELETE /identity/v3/projects/57d971df6ca64a209bbecf001aea602f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:22.610519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6267c78d0e3c48b2aed6c4d7ea3cc8ad'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.611184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6267c78d0e3c48b2aed6c4d7ea3cc8ad'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.612411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:22.613795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b8d6626b-ea72-43ce-a992-db2221c256eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6267c78d0e3c48b2aed6c4d7ea3cc8ad'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.615578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:22.657918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1014/2007] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:22 2022] DELETE /identity/v3/users/6267c78d0e3c48b2aed6c4d7ea3cc8ad => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:20:22.662002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.662359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.662703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.686903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:22.688947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:22.719840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb066705-05a1-4b8a-819c-5e23462e29a0 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:22.732227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:22.770607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1015/2008] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:20:22.782842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:22.785217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:22.801209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:22.802260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.802629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.802970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] PATH_INFO: `/v3/users/ce7f0426b0c947e2a8e4c838c0be7f24` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.804706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7f0426b0c947e2a8e4c838c0be7f24)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:22.806715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:22.875607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7f0426b0c947e2a8e4c838c0be7f24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.885092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7f0426b0c947e2a8e4c838c0be7f24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.885758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531a8cd-e094-44f3-966f-36867b061d64 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:22.889521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a15ab-d5d8-454a-8920-2cad5c22ddab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:22.890168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a15ab-d5d8-454a-8920-2cad5c22ddab service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:22.890456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a15ab-d5d8-454a-8920-2cad5c22ddab service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:22.890752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a15ab-d5d8-454a-8920-2cad5c22ddab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:22.893334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7f0426b0c947e2a8e4c838c0be7f24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.893997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7f0426b0c947e2a8e4c838c0be7f24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.894572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7f0426b0c947e2a8e4c838c0be7f24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:22.901792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a15ab-d5d8-454a-8920-2cad5c22ddab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:22.903520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a15ab-d5d8-454a-8920-2cad5c22ddab service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:22.915776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 994/2009] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:22 2022] DELETE /identity/v3/users/ce7f0426b0c947e2a8e4c838c0be7f24 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:22.916421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1016/2010] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:23.892885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:23.895073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:23.980030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c10ddf-56c0-450d-9ec0-9d466038d0b8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:23.984973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:23.985853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:23.986134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:23.986416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] PATH_INFO: `/v3/projects/4aedf5a132df4ffd9bfa8be445e008fa` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:23.987632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aedf5a132df4ffd9bfa8be445e008fa)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:23.989663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.042058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:24.057817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aedf5a132df4ffd9bfa8be445e008fa', 'request_id': 'req-d4aa45fe-e569-43dd-8a56-1138e98c9892'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.066342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aedf5a132df4ffd9bfa8be445e008fa', 'request_id': 'req-d4aa45fe-e569-43dd-8a56-1138e98c9892'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.074402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aedf5a132df4ffd9bfa8be445e008fa', 'request_id': 'req-d4aa45fe-e569-43dd-8a56-1138e98c9892'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.092097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] The token cache is being invalidate because project 4aedf5a132df4ffd9bfa8be445e008fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:24.093386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.095202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.098268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 995/2011] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:23 2022] DELETE /identity/v3/projects/4aedf5a132df4ffd9bfa8be445e008fa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:24.103942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cba71-c584-49f5-a5d9-4a73c5bc0a4e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.104577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cba71-c584-49f5-a5d9-4a73c5bc0a4e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.104577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cba71-c584-49f5-a5d9-4a73c5bc0a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.126723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f65cba71-c584-49f5-a5d9-4a73c5bc0a4e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:24.128047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f65cba71-c584-49f5-a5d9-4a73c5bc0a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:24.204252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:24.207643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:24.212384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1017/2012] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:24.220911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488a22a-744d-4d8e-b148-e1fa48c65902 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.220911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488a22a-744d-4d8e-b148-e1fa48c65902 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.221181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488a22a-744d-4d8e-b148-e1fa48c65902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.250562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5488a22a-744d-4d8e-b148-e1fa48c65902 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:24.251045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5488a22a-744d-4d8e-b148-e1fa48c65902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:24.256811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa45fe-e569-43dd-8a56-1138e98c9892 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:24.260180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1018/2013] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:24.265412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488a22a-744d-4d8e-b148-e1fa48c65902 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:24.273050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:24.275186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.277434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.277809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] PATH_INFO: `/v3/projects/6b3bcc7e2a2a4ca8a0d6a0e71c393723` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.282787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3bcc7e2a2a4ca8a0d6a0e71c393723)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:24.285922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488a22a-744d-4d8e-b148-e1fa48c65902 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:24.290746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.299078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:24.299770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.300065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.300334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.301110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:20:24.301422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:24.303049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.313421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:24.334437 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1019/2014] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:20:24.339291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:24.363038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb542c4-3766-4012-b060-d60a580196a7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:24.373052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:24.373796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.374090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.374598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.375859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:24.379038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.420067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3bcc7e2a2a4ca8a0d6a0e71c393723', 'request_id': 'req-267d5f1e-66f7-4563-8cf8-743ebe0ad120'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.427936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1020/2015] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:24 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:24.429391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:24.445029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3bcc7e2a2a4ca8a0d6a0e71c393723', 'request_id': 'req-267d5f1e-66f7-4563-8cf8-743ebe0ad120'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.454148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3bcc7e2a2a4ca8a0d6a0e71c393723', 'request_id': 'req-267d5f1e-66f7-4563-8cf8-743ebe0ad120'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.476318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] The token cache is being invalidate because project 6b3bcc7e2a2a4ca8a0d6a0e71c393723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:24.478164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.479107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:24.484197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 996/2016] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:24 2022] DELETE /identity/v3/projects/6b3bcc7e2a2a4ca8a0d6a0e71c393723 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:20:24.575249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9987602-139d-4034-b3f5-955d30421a54 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:24.584372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:24.585530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.586032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.586543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.587792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:24.590109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.632206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1021/2017] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:24 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:24.636125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:24.637898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:24.749337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d5f1e-66f7-4563-8cf8-743ebe0ad120 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:24.755305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:24.755995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.756328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.756656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.757395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:24.759516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.782552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 997/2018] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:20:24.786898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:24.802508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec399027-c0f1-4cd2-b0b8-33a6bb30cef7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:24.809292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afb83c-9540-45d4-a373-b7bd00debc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:24.809292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afb83c-9540-45d4-a373-b7bd00debc75 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.809720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afb83c-9540-45d4-a373-b7bd00debc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.809907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afb83c-9540-45d4-a373-b7bd00debc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.810719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afb83c-9540-45d4-a373-b7bd00debc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:24.812514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afb83c-9540-45d4-a373-b7bd00debc75 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.831448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1022/2019] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:24.835332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:24.853153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565f61-0813-4ad4-be30-a9782a55eeb6 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:24.868252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:24.869502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.870147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.870630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] PATH_INFO: `/v3/projects/13c1170fcc784d1cbbfd852ab792bb3b/users/e11271dddf4f419d975c9e1f4f8dd89a/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.903728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c1170fcc784d1cbbfd852ab792bb3b, user_id=e11271dddf4f419d975c9e1f4f8dd89a, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:24.907295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:24.927200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 998/2020] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:24 2022] PUT /identity/v3/projects/13c1170fcc784d1cbbfd852ab792bb3b/users/e11271dddf4f419d975c9e1f4f8dd89a/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:24.939566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff734-06d9-43b3-8f29-eb6722be2556 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:24.939858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff734-06d9-43b3-8f29-eb6722be2556 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:24.940133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff734-06d9-43b3-8f29-eb6722be2556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:24.998464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-106ff734-06d9-43b3-8f29-eb6722be2556 None None] MFA Rules not processed for user `e11271dddf4f419d975c9e1f4f8dd89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:25.001745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-106ff734-06d9-43b3-8f29-eb6722be2556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:25.082413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1023/2021] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:25.084974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:25.106500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:25.165439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885812b-e0ac-4dcb-b3cd-f7cd0db36f13 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:25.171121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:25.172608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.173097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.173545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.174435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:25.176541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:25.210420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 999/2022] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:25 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:20:25.214066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ff734-06d9-43b3-8f29-eb6722be2556 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:25.225686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ff734-06d9-43b3-8f29-eb6722be2556 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:25.273227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:25.304078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:25.304078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.304078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.304078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.304417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:25.304417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:25.325163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1024/2023] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:25 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:20:25.329266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:25.340804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19458a2-029d-4fb7-8e1b-799ae0c4f759 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:25.346222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:25.346921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.347203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.347474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.348197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:25.349860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:25.361726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1000/2024] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:25.366007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:25.378195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b24719-037d-45dd-83b7-45e850433dc7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:25.383105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:25.383993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.384276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.384578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] PATH_INFO: `/v3/projects/bd66e73a111e468c97e5dd19a3d62640/users/38290cacee4446df828053f852efed62/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.404769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd66e73a111e468c97e5dd19a3d62640, user_id=38290cacee4446df828053f852efed62, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:25.407987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:25.424215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1025/2025] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:25 2022] PUT /identity/v3/projects/bd66e73a111e468c97e5dd19a3d62640/users/38290cacee4446df828053f852efed62/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:25.428113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:25.439164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bee230-69f0-444b-810c-24d6f62b104f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:25.503480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef341c-7601-4088-9791-3809f4707d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:25.503848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef341c-7601-4088-9791-3809f4707d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.504136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef341c-7601-4088-9791-3809f4707d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.504413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef341c-7601-4088-9791-3809f4707d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.505319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef341c-7601-4088-9791-3809f4707d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:25.506973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef341c-7601-4088-9791-3809f4707d3f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:25.519050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1001/2026] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:25.523101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:25.534494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ed89a-fc2b-43b8-8655-2feb4108e934 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:25.539316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:25.540295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.540687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.541033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] PATH_INFO: `/v3/projects/bd66e73a111e468c97e5dd19a3d62640/users/38290cacee4446df828053f852efed62/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.543146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd66e73a111e468c97e5dd19a3d62640, user_id=38290cacee4446df828053f852efed62, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:25.546321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:25.558281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1026/2027] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:25 2022] PUT /identity/v3/projects/bd66e73a111e468c97e5dd19a3d62640/users/38290cacee4446df828053f852efed62/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:25.568326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b026-259b-4884-b643-ff76c59dcd6b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.568645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b026-259b-4884-b643-ff76c59dcd6b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.568924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b026-259b-4884-b643-ff76c59dcd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.598957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-0b23b026-259b-4884-b643-ff76c59dcd6b None None] MFA Rules not processed for user `38290cacee4446df828053f852efed62`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:25.600167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0b23b026-259b-4884-b643-ff76c59dcd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:25.677588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1002/2028] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:25.689719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:25.691471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:25.764441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c362f7-9137-431e-8023-0c532be1c0cc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:25.769482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:25.770137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:25.770420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:25.770696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:25.839497 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:25.841273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:25.852898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1027/2029] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:31.592813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b026-259b-4884-b643-ff76c59dcd6b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:31.603718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b026-259b-4884-b643-ff76c59dcd6b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:31.609602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e05b8-8b1f-4e0a-a26f-1af705a27517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:31.611265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e05b8-8b1f-4e0a-a26f-1af705a27517 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:31.611265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e05b8-8b1f-4e0a-a26f-1af705a27517 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:31.611265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e05b8-8b1f-4e0a-a26f-1af705a27517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:31.625421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e05b8-8b1f-4e0a-a26f-1af705a27517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:31.627434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e05b8-8b1f-4e0a-a26f-1af705a27517 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:31.632399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:31.658992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1003/2030] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:35.847607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:36.856579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:36.919560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8109a2-076d-4416-bea9-a3ce9616c9fe service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:36.924519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:36.925404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:36.925688 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:36.925962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] PATH_INFO: `/v3/users/e11271dddf4f419d975c9e1f4f8dd89a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:36.927154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11271dddf4f419d975c9e1f4f8dd89a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:36.929387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:36.977408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11271dddf4f419d975c9e1f4f8dd89a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:36.984948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11271dddf4f419d975c9e1f4f8dd89a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:36.991747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11271dddf4f419d975c9e1f4f8dd89a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:36.992378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11271dddf4f419d975c9e1f4f8dd89a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:36.993150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11271dddf4f419d975c9e1f4f8dd89a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.011751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1028/2031] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:36 2022] DELETE /identity/v3/users/e11271dddf4f419d975c9e1f4f8dd89a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:37.016695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:37.017231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:37.017231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:37.035888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:37.037131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:37.097318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1004/2032] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:20:37.106815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:37.108233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:37.173272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6d2b6-0ae3-4f50-9641-a309adc4fb34 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:37.176092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:37.176780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:37.177069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:37.177340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:37.187098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:37.188537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:37.198035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1029/2033] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:37.473373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:37.475304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:37.494113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e7793-0519-4e55-b710-53d74a5b2d46 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:37.498726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:37.499594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:37.499878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:37.500250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] PATH_INFO: `/v3/projects/13c1170fcc784d1cbbfd852ab792bb3b` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:37.501733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c1170fcc784d1cbbfd852ab792bb3b)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:37.503645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:37.551638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c1170fcc784d1cbbfd852ab792bb3b', 'request_id': 'req-50d82851-4fdf-4d5a-9741-7749bc6a50dd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.559014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c1170fcc784d1cbbfd852ab792bb3b', 'request_id': 'req-50d82851-4fdf-4d5a-9741-7749bc6a50dd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.564533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c1170fcc784d1cbbfd852ab792bb3b', 'request_id': 'req-50d82851-4fdf-4d5a-9741-7749bc6a50dd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.575534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] The token cache is being invalidate because project 13c1170fcc784d1cbbfd852ab792bb3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:37.575834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.576467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.578689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1005/2034] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:37 2022] DELETE /identity/v3/projects/13c1170fcc784d1cbbfd852ab792bb3b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:37.582524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:37.592032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b306c45-0ddc-4f60-b861-bf74d31aa33b service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:37.611449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:37.652798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:37.653667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:37.653950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:37.654229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] PATH_INFO: `/v3/users/38290cacee4446df828053f852efed62` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:37.655410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38290cacee4446df828053f852efed62)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:37.657967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:37.706754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38290cacee4446df828053f852efed62'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.713527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38290cacee4446df828053f852efed62'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.720969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38290cacee4446df828053f852efed62'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.721603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38290cacee4446df828053f852efed62'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.722214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292141c5-f934-4cd1-ab2e-3c7f6a964456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38290cacee4446df828053f852efed62'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:37.741833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1030/2035] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:37 2022] DELETE /identity/v3/users/38290cacee4446df828053f852efed62 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:38.110568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.112820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:38.152877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d82851-4fdf-4d5a-9741-7749bc6a50dd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.213804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.214675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.214984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.215261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] PATH_INFO: `/v3/projects/bd66e73a111e468c97e5dd19a3d62640` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.216458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd66e73a111e468c97e5dd19a3d62640)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.218376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.252780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:38.269990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd66e73a111e468c97e5dd19a3d62640', 'request_id': 'req-627f2ea8-95d5-4745-96ce-f91538a47386'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:38.277016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd66e73a111e468c97e5dd19a3d62640', 'request_id': 'req-627f2ea8-95d5-4745-96ce-f91538a47386'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:38.282516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd66e73a111e468c97e5dd19a3d62640', 'request_id': 'req-627f2ea8-95d5-4745-96ce-f91538a47386'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:38.293370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] The token cache is being invalidate because project bd66e73a111e468c97e5dd19a3d62640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:38.293658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:38.294281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-627f2ea8-95d5-4745-96ce-f91538a47386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:38.295820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1006/2036] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:38 2022] DELETE /identity/v3/projects/bd66e73a111e468c97e5dd19a3d62640 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:38.302340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66f692-eea3-4c70-b50d-483093213246 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.302800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66f692-eea3-4c70-b50d-483093213246 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.302888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66f692-eea3-4c70-b50d-483093213246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.321671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cf66f692-eea3-4c70-b50d-483093213246 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:38.322888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cf66f692-eea3-4c70-b50d-483093213246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:38.381026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1031/2037] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:38.390343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3178d-0fec-4cc8-a4bc-88571aeb00e6 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.390635 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3178d-0fec-4cc8-a4bc-88571aeb00e6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.390917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3178d-0fec-4cc8-a4bc-88571aeb00e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.408828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9dd3178d-0fec-4cc8-a4bc-88571aeb00e6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:38.410074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9dd3178d-0fec-4cc8-a4bc-88571aeb00e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:38.418253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1007/2038] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:38.422925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66f692-eea3-4c70-b50d-483093213246 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.433114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66f692-eea3-4c70-b50d-483093213246 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.438197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.438845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.439131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.439407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.440131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:20:38.440438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.442063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.453440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1032/2039] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:20:38.457598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3178d-0fec-4cc8-a4bc-88571aeb00e6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.466779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3178d-0fec-4cc8-a4bc-88571aeb00e6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.471499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.472146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.472431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.472754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.473442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.475328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.506235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1008/2040] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:38 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:38.507818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.518887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323b4d2-f1a2-4193-8623-cea38cc53e8c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.545433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:38.577553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.578279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.578562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.578835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.579516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.581456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.601200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1033/2041] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:38 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:38.605396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.617711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dd8fa-9417-407b-88c6-0d763519b495 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.623228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.623919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.624207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.624488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.625271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.626895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.638049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1009/2042] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:20:38.641995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.651850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc02ee4-6312-44bc-92ff-0ecf97eb794f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.657082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983ee8f-3dd7-4968-8a77-6b8a1ec253f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.657732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983ee8f-3dd7-4968-8a77-6b8a1ec253f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.658012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983ee8f-3dd7-4968-8a77-6b8a1ec253f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.658286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983ee8f-3dd7-4968-8a77-6b8a1ec253f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.659011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983ee8f-3dd7-4968-8a77-6b8a1ec253f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.660714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983ee8f-3dd7-4968-8a77-6b8a1ec253f1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.671011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1034/2043] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:38.675717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.686155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d671a7-1655-439a-a590-cc0dfedfe197 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.690775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.691654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.691933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.692208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] PATH_INFO: `/v3/projects/148861fd67cb4d0380fce5b62a8a7318/users/ef5d8eee69c549a9bb994b1046b9250d/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.712825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] RBAC: Authorizing `identity:create_grant(project_id=148861fd67cb4d0380fce5b62a8a7318, user_id=ef5d8eee69c549a9bb994b1046b9250d, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.715543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.732325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1010/2044] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:38 2022] PUT /identity/v3/projects/148861fd67cb4d0380fce5b62a8a7318/users/ef5d8eee69c549a9bb994b1046b9250d/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:38.744340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.744656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.744998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.773338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] MFA Rules not processed for user `ef5d8eee69c549a9bb994b1046b9250d`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:38.774642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:38.859605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1035/2045] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:38.870684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:38.872429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:38.930344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3156-8c7b-4ede-8ac8-1771474e386a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:38.933998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:38.934646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:38.934928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:38.935216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:38.944256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:38.945734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:38.960506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1011/2046] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:20:48.850696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:48.955837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:49.073946 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:49.075939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:49.156033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39c94-1dcf-4523-bf17-a6bc12da428c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:49.160749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:49.161621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.161908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.162190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] PATH_INFO: `/v3/users/592836cc56ed4334bb46890a74460307` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.163410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=592836cc56ed4334bb46890a74460307)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:49.165393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:49.219794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592836cc56ed4334bb46890a74460307'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.227055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592836cc56ed4334bb46890a74460307'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.233885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592836cc56ed4334bb46890a74460307'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.234627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592836cc56ed4334bb46890a74460307'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.235234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b25d83f5-3306-42a9-a1ad-dc0e3895ad1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592836cc56ed4334bb46890a74460307'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.253120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1036/2047] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:20:49 2022] DELETE /identity/v3/users/592836cc56ed4334bb46890a74460307 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:49.432488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:49.444887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcc434-ee2a-4d7e-9cb4-18c5f2183255 service glance] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:49.503128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:49.503990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.504270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.504549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] PATH_INFO: `/v3/projects/5c235d441df94f80a393e5639413d4d9` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.505772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c235d441df94f80a393e5639413d4d9)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:49.507706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:49.554992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c235d441df94f80a393e5639413d4d9', 'request_id': 'req-b3037ccb-0d3d-4c17-b73e-12149ac98c83'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.563328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c235d441df94f80a393e5639413d4d9', 'request_id': 'req-b3037ccb-0d3d-4c17-b73e-12149ac98c83'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.570638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c235d441df94f80a393e5639413d4d9', 'request_id': 'req-b3037ccb-0d3d-4c17-b73e-12149ac98c83'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.582545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] The token cache is being invalidate because project 5c235d441df94f80a393e5639413d4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:20:49.582871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.583523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b3037ccb-0d3d-4c17-b73e-12149ac98c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:20:49.585094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1012/2048] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:20:49 2022] DELETE /identity/v3/projects/5c235d441df94f80a393e5639413d4d9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:49.590109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcff52-c830-4e13-9499-71370b7af849 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.590512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcff52-c830-4e13-9499-71370b7af849 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.590964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcff52-c830-4e13-9499-71370b7af849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.608530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-abfcff52-c830-4e13-9499-71370b7af849 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:49.610371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-abfcff52-c830-4e13-9499-71370b7af849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:49.688071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1037/2049] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:49.695837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a01471-1659-4980-9c8b-3e1b03ab40d9 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.696132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a01471-1659-4980-9c8b-3e1b03ab40d9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.696416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a01471-1659-4980-9c8b-3e1b03ab40d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.719521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-69a01471-1659-4980-9c8b-3e1b03ab40d9 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:49.720791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-69a01471-1659-4980-9c8b-3e1b03ab40d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:49.728845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1013/2050] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:49.733017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcff52-c830-4e13-9499-71370b7af849 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:49.743995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcff52-c830-4e13-9499-71370b7af849 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:49.748641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:49.749302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.749587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.750460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.751192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:20:49.751500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:49.753272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:49.765045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1038/2051] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:20:49.769453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a01471-1659-4980-9c8b-3e1b03ab40d9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:49.779159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a01471-1659-4980-9c8b-3e1b03ab40d9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:49.784151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:49.784837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.785116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.785388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.786074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:49.787988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:49.816591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1014/2052] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:20:49 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:49.820123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:49.837373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac288b4f-1eb6-4f58-9815-991240e0a6bc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:49.841543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:49.907083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:49.907767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.908053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.908330 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.909065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:49.910991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:49.937619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1039/2053] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:20:49 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:20:49.939399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:49.950392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bab47-261d-420e-9845-e46255723b50 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:49.955063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:49.955702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.955980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.956503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.957001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:49.958598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:49.970522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1015/2054] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:49.974351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:49.985058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e6d2-81f9-4902-b39a-aa1a49251008 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:49.989722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf2b13-ced8-49eb-b120-aeee5fce895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:49.990371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf2b13-ced8-49eb-b120-aeee5fce895c admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:49.990657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf2b13-ced8-49eb-b120-aeee5fce895c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:49.990958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf2b13-ced8-49eb-b120-aeee5fce895c admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:49.991679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf2b13-ced8-49eb-b120-aeee5fce895c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:49.993337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf2b13-ced8-49eb-b120-aeee5fce895c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:50.003115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1040/2055] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:20:50.006755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:50.016465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621c915-33bb-4d17-80ab-79143925d5fb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:50.021190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:50.022067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:50.022349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:50.022622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] PATH_INFO: `/v3/projects/8a56765f6f92417cbf8b619ef5e9cb78/users/e0ce482b251d4f9f91cfbd355231e478/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:50.037603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a56765f6f92417cbf8b619ef5e9cb78, user_id=e0ce482b251d4f9f91cfbd355231e478, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:50.040267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:50.054671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1016/2056] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:20:50 2022] PUT /identity/v3/projects/8a56765f6f92417cbf8b619ef5e9cb78/users/e0ce482b251d4f9f91cfbd355231e478/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:20:50.065871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d21d5-132b-4325-8fee-f40a845b0cf8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:50.066167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d21d5-132b-4325-8fee-f40a845b0cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:50.066448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d21d5-132b-4325-8fee-f40a845b0cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:50.096624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-5a0d21d5-132b-4325-8fee-f40a845b0cf8 None None] MFA Rules not processed for user `e0ce482b251d4f9f91cfbd355231e478`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:20:50.097853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5a0d21d5-132b-4325-8fee-f40a845b0cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:50.164484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1041/2057] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:20:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:20:50.175748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:20:50.177537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:20:50.212954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:20:50.262625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a681a2-dcbc-4cfc-95a9-76c79892cc48 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:20:50.268666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011f816-da0f-422c-85d7-fa005b1df807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:20:50.269326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011f816-da0f-422c-85d7-fa005b1df807 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:20:50.269604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011f816-da0f-422c-85d7-fa005b1df807 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:20:50.269878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011f816-da0f-422c-85d7-fa005b1df807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:20:50.278010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011f816-da0f-422c-85d7-fa005b1df807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:20:50.279580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011f816-da0f-422c-85d7-fa005b1df807 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:20:50.291206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1017/2058] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:20:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:00.161372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:00.285938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=22 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:08.015936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d21d5-132b-4325-8fee-f40a845b0cf8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:08.119322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d21d5-132b-4325-8fee-f40a845b0cf8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:08.123841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:08.124732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:08.125013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:08.125286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] PATH_INFO: `/v3/users/e0ce482b251d4f9f91cfbd355231e478` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:08.126544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ce482b251d4f9f91cfbd355231e478)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:08.128443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:08.173969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ce482b251d4f9f91cfbd355231e478'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.180900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ce482b251d4f9f91cfbd355231e478'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.187902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ce482b251d4f9f91cfbd355231e478'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.188540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ce482b251d4f9f91cfbd355231e478'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.189185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ce482b251d4f9f91cfbd355231e478'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.206486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1042/2059] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:08 2022] DELETE /identity/v3/users/e0ce482b251d4f9f91cfbd355231e478 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:08.210975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:08.211455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:08.211561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:08.257277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:08.258452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:08.315035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1018/2060] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:08.324464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:08.325882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:08.370335 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:08.424870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4ca9a-92db-48ee-b31b-cb918c11dde5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:08.428470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd9d82-eb9b-4c53-8ab3-2be89cc5120d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:08.429445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd9d82-eb9b-4c53-8ab3-2be89cc5120d service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:08.429738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd9d82-eb9b-4c53-8ab3-2be89cc5120d service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:08.430012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd9d82-eb9b-4c53-8ab3-2be89cc5120d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:08.439392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd9d82-eb9b-4c53-8ab3-2be89cc5120d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:08.440841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd9d82-eb9b-4c53-8ab3-2be89cc5120d service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:08.450395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1043/2061] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:08.728460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:08.729940 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:08.741566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444c609-74b8-42db-b7cf-08d3653d4165 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:08.746288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:08.747209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:08.747493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:08.747922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] PATH_INFO: `/v3/projects/8a56765f6f92417cbf8b619ef5e9cb78` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:08.749156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a56765f6f92417cbf8b619ef5e9cb78)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:08.751216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:08.803112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a56765f6f92417cbf8b619ef5e9cb78', 'request_id': 'req-7d8d9b47-2144-432c-8cad-82f8cd8500a2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.811734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a56765f6f92417cbf8b619ef5e9cb78', 'request_id': 'req-7d8d9b47-2144-432c-8cad-82f8cd8500a2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.818211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a56765f6f92417cbf8b619ef5e9cb78', 'request_id': 'req-7d8d9b47-2144-432c-8cad-82f8cd8500a2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.829184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] The token cache is being invalidate because project 8a56765f6f92417cbf8b619ef5e9cb78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:08.830088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.830733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7d8d9b47-2144-432c-8cad-82f8cd8500a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:08.832258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1019/2062] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:08 2022] DELETE /identity/v3/projects/8a56765f6f92417cbf8b619ef5e9cb78 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:08.836983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c95ee-5a0b-4acc-93b5-ef45e9243947 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:08.837371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c95ee-5a0b-4acc-93b5-ef45e9243947 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:08.837739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c95ee-5a0b-4acc-93b5-ef45e9243947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:08.874110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a68c95ee-5a0b-4acc-93b5-ef45e9243947 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:08.875299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a68c95ee-5a0b-4acc-93b5-ef45e9243947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:08.938080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1044/2063] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:08.946404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716fedb-721f-4778-abf1-2f4c3821be05 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:08.946693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716fedb-721f-4778-abf1-2f4c3821be05 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:08.946969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716fedb-721f-4778-abf1-2f4c3821be05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:08.964829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-4716fedb-721f-4778-abf1-2f4c3821be05 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:08.966074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4716fedb-721f-4778-abf1-2f4c3821be05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:08.973918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1020/2064] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:08.977959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c95ee-5a0b-4acc-93b5-ef45e9243947 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:08.988629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c95ee-5a0b-4acc-93b5-ef45e9243947 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:08.993392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:08.994083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:08.994369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:08.994643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:08.995402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:08.995706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:08.997328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.010029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1045/2065] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:21:09.014133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716fedb-721f-4778-abf1-2f4c3821be05 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.023480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716fedb-721f-4778-abf1-2f4c3821be05 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.028170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.028857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.029136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.029407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.030170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.032089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.057795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1021/2066] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:09 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:09.062130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.072766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4d2ce-40fb-4121-a0a4-f7cb38ac56f7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.129470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.130131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.130415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.130691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.131391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.133329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.152408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1046/2067] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:09 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:09.156498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.166851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c7960-ee4e-47d1-8082-bc104e11768b admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.171450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.172115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.172400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.172718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.173952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.175544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.187034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1022/2068] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:09.191435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.201294 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12c9f4-53da-466c-9657-4862723e89a4 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.205751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e839fb-960b-4aa7-9bd1-e1e6821539e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.206396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e839fb-960b-4aa7-9bd1-e1e6821539e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.206677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e839fb-960b-4aa7-9bd1-e1e6821539e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.206951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e839fb-960b-4aa7-9bd1-e1e6821539e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.207677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e839fb-960b-4aa7-9bd1-e1e6821539e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.209284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e839fb-960b-4aa7-9bd1-e1e6821539e2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.210495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:09.221164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1047/2069] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:09.224980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.234621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cda2f-0d7e-493d-afea-f7492747acb7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.240778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.241939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.242422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.242949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] PATH_INFO: `/v3/projects/cebaefb1297d42fca6e2a514a279877c/users/5153586e6b3e47d79e034bc774e1adab/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.259919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebaefb1297d42fca6e2a514a279877c, user_id=5153586e6b3e47d79e034bc774e1adab, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.263434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.265638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:09.278586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1023/2070] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:09 2022] PUT /identity/v3/projects/cebaefb1297d42fca6e2a514a279877c/users/5153586e6b3e47d79e034bc774e1adab/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:09.289373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d51fb-c581-46d2-8535-fe9db464590f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.289664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d51fb-c581-46d2-8535-fe9db464590f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.289943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d51fb-c581-46d2-8535-fe9db464590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.316196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-469d51fb-c581-46d2-8535-fe9db464590f None None] MFA Rules not processed for user `5153586e6b3e47d79e034bc774e1adab`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:09.317438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-469d51fb-c581-46d2-8535-fe9db464590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:09.379581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1048/2071] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:09.384098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.444287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87f1c-9cad-4fb6-8054-f20d371c0f72 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.448932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.449589 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.449871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.450148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.450847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.452825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.477609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1024/2072] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:09 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:09.481552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d51fb-c581-46d2-8535-fe9db464590f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.492917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d51fb-c581-46d2-8535-fe9db464590f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.549817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.550489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.550771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.551046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.551752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.553947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.573275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1049/2073] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:09 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:09.577292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.588223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d487a54-0c5e-44d9-a048-a1c42b92db88 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.593612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.593612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.593907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.594183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.594904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.597165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.608245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1025/2074] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:09.612402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.622504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8dcff-1bbe-4dd6-a2df-151e4e9e9c02 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.636738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.637644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.637929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.638206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] PATH_INFO: `/v3/projects/e45ea429529d4927bd7adfa43b9caeec/users/c99d667f49d34885ac761a603793b351/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.655465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45ea429529d4927bd7adfa43b9caeec, user_id=c99d667f49d34885ac761a603793b351, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.658838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.672748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1050/2075] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:09 2022] PUT /identity/v3/projects/e45ea429529d4927bd7adfa43b9caeec/users/c99d667f49d34885ac761a603793b351/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:09.677656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.687699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761eb99-5c4d-4fef-94e3-9c86f2a225e9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.742902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c24e1-aa4e-4fa4-a907-6b4154aa5c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.743491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c24e1-aa4e-4fa4-a907-6b4154aa5c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.743776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c24e1-aa4e-4fa4-a907-6b4154aa5c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.744053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c24e1-aa4e-4fa4-a907-6b4154aa5c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.744805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c24e1-aa4e-4fa4-a907-6b4154aa5c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.746398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c24e1-aa4e-4fa4-a907-6b4154aa5c90 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.755946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1026/2076] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:09.760081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.770871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aa1c0-514d-4061-bd87-975a235c15ac admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.775791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.776754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.777067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.777345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] PATH_INFO: `/v3/projects/e45ea429529d4927bd7adfa43b9caeec/users/c99d667f49d34885ac761a603793b351/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.779318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45ea429529d4927bd7adfa43b9caeec, user_id=c99d667f49d34885ac761a603793b351, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.781997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.792805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1051/2077] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:09 2022] PUT /identity/v3/projects/e45ea429529d4927bd7adfa43b9caeec/users/c99d667f49d34885ac761a603793b351/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:09.804139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.804428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.804737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.831155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] MFA Rules not processed for user `c99d667f49d34885ac761a603793b351`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:09.832346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:09.881727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:09.901700 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1027/2078] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:09.913380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:09.915087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:09.956633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:09.973602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca612c-f7a1-4ad6-82ac-c88ff374d255 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:09.977245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:09.977890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:09.978173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:09.978449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:09.987652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:09.989115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:09.998642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1052/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:10.077510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:10.078885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:10.091272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2d720-fa2d-4a44-916f-89f726c5187e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:10.094878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ad63-0497-4b45-a3d4-4e947737823e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:10.095534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ad63-0497-4b45-a3d4-4e947737823e service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:10.095816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ad63-0497-4b45-a3d4-4e947737823e service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:10.096090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ad63-0497-4b45-a3d4-4e947737823e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:10.152070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ad63-0497-4b45-a3d4-4e947737823e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:10.153532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ad63-0497-4b45-a3d4-4e947737823e service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:10.162229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1028/2080] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:19.994299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=9 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:20.158315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=33 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:23.079768 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:23.141576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3c65-b4f6-4ef3-b4f6-f70b58854f91 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:23.146266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:23.147133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:23.147419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:23.147700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] PATH_INFO: `/v3/users/5153586e6b3e47d79e034bc774e1adab` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:23.148914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5153586e6b3e47d79e034bc774e1adab)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:23.150833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:23.197784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5153586e6b3e47d79e034bc774e1adab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.205333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5153586e6b3e47d79e034bc774e1adab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.212136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5153586e6b3e47d79e034bc774e1adab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.212797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5153586e6b3e47d79e034bc774e1adab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.213405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5153586e6b3e47d79e034bc774e1adab'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.230225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1053/2081] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:23 2022] DELETE /identity/v3/users/5153586e6b3e47d79e034bc774e1adab => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:23.234819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:23.235296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:23.235397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:23.252603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:23.253737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:23.314666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1029/2082] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:23.324757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:23.326232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:23.382442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e65d6e-2818-4df7-9635-7e973610a193 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:23.386048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:23.386681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:23.386961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:23.387233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:23.396127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:23.397562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:23.406925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1054/2083] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:23.721560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:23.723895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:23.740929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de804e-044c-4a4f-9fce-8ba59b0f967f None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:23.745571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:23.746438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:23.746719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:23.747001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] PATH_INFO: `/v3/projects/cebaefb1297d42fca6e2a514a279877c` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:23.748192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebaefb1297d42fca6e2a514a279877c)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:23.750135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:23.799485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebaefb1297d42fca6e2a514a279877c', 'request_id': 'req-7b5db2f4-1538-4801-924e-55a7c4bdfc83'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.810292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebaefb1297d42fca6e2a514a279877c', 'request_id': 'req-7b5db2f4-1538-4801-924e-55a7c4bdfc83'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.822385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebaefb1297d42fca6e2a514a279877c', 'request_id': 'req-7b5db2f4-1538-4801-924e-55a7c4bdfc83'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.835286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] The token cache is being invalidate because project cebaefb1297d42fca6e2a514a279877c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:23.835676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.836385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.837996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1030/2084] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:23 2022] DELETE /identity/v3/projects/cebaefb1297d42fca6e2a514a279877c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:23.842073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:23.853397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a036fea-0af5-4254-9245-69a50995121b service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:23.872655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:23.913628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:23.914491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:23.914771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:23.915044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] PATH_INFO: `/v3/users/c99d667f49d34885ac761a603793b351` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:23.916220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99d667f49d34885ac761a603793b351)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:23.918158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:23.963886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99d667f49d34885ac761a603793b351'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.973938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99d667f49d34885ac761a603793b351'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.981105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99d667f49d34885ac761a603793b351'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.981736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99d667f49d34885ac761a603793b351'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.982337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-2363661f-752a-4d52-bafc-f59d49fb6cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99d667f49d34885ac761a603793b351'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:23.999061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1055/2085] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:23 2022] DELETE /identity/v3/users/c99d667f49d34885ac761a603793b351 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:24.265142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:24.266673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:24.280363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5db2f4-1538-4801-924e-55a7c4bdfc83 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:24.337418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:24.338273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.338552 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.338826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] PATH_INFO: `/v3/projects/e45ea429529d4927bd7adfa43b9caeec` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.339981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45ea429529d4927bd7adfa43b9caeec)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:24.341896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:24.370218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:24.388062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45ea429529d4927bd7adfa43b9caeec', 'request_id': 'req-eeea287c-559f-4bd8-b9a9-e9981f24b115'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:24.395798 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45ea429529d4927bd7adfa43b9caeec', 'request_id': 'req-eeea287c-559f-4bd8-b9a9-e9981f24b115'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:24.401249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45ea429529d4927bd7adfa43b9caeec', 'request_id': 'req-eeea287c-559f-4bd8-b9a9-e9981f24b115'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:24.411951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] The token cache is being invalidate because project e45ea429529d4927bd7adfa43b9caeec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:24.412232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:24.412876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-eeea287c-559f-4bd8-b9a9-e9981f24b115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:24.414385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1031/2086] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:24 2022] DELETE /identity/v3/projects/e45ea429529d4927bd7adfa43b9caeec => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:24.419217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16152b3-9e1f-4e59-ad50-e95bc02ba59d None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.419680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16152b3-9e1f-4e59-ad50-e95bc02ba59d None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.419780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16152b3-9e1f-4e59-ad50-e95bc02ba59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.447380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d16152b3-9e1f-4e59-ad50-e95bc02ba59d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:24.448577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d16152b3-9e1f-4e59-ad50-e95bc02ba59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:24.507395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1056/2087] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:24.514956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aba840-473d-4b92-97e2-9c059e7e128e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.515249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aba840-473d-4b92-97e2-9c059e7e128e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.515530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aba840-473d-4b92-97e2-9c059e7e128e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.535221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-04aba840-473d-4b92-97e2-9c059e7e128e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:24.536415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-04aba840-473d-4b92-97e2-9c059e7e128e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:24.544496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1032/2088] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:24.552659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16152b3-9e1f-4e59-ad50-e95bc02ba59d None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:24.558661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16152b3-9e1f-4e59-ad50-e95bc02ba59d None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:24.563152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:24.563798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.564076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.564350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.565092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:24.565393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:24.567479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:24.578834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1057/2089] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:21:24.583445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aba840-473d-4b92-97e2-9c059e7e128e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:24.592580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aba840-473d-4b92-97e2-9c059e7e128e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:24.597119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:24.597758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.598045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.598315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.599000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:24.600930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:24.626387 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1033/2090] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:24 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:24.631043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:24.641240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487818e-1c15-42b1-b1a7-2de464c0362b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:24.657607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:24.699918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:24.700627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.700913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.701188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.701874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:24.703785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:24.724439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1058/2091] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:24 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:24.728375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:24.739341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d4bcf-c440-46af-af52-2cec6a6b5ae3 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:24.743965 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:24.745085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.745085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.745315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.745961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:24.747579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:24.759125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1034/2092] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:24.763129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:24.773670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9c833-5bb5-49b1-b05b-a95ec1c7b68a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:24.778564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12635b34-40d1-40ec-97fe-d2ac4fcbf937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:24.779299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12635b34-40d1-40ec-97fe-d2ac4fcbf937 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.779686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12635b34-40d1-40ec-97fe-d2ac4fcbf937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.780023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12635b34-40d1-40ec-97fe-d2ac4fcbf937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.781019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12635b34-40d1-40ec-97fe-d2ac4fcbf937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:24.782616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12635b34-40d1-40ec-97fe-d2ac4fcbf937 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:24.793139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1059/2093] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:24.797609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:24.808628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ffb73-2dbf-4028-a50c-9dedfcec16f4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:24.814045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:24.814503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.814846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.815185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] PATH_INFO: `/v3/projects/f72843e62e2643178d716c409ffd22ac/users/c4fe8d0e1ec64d6f9b03def5173d2615/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.836785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f72843e62e2643178d716c409ffd22ac, user_id=c4fe8d0e1ec64d6f9b03def5173d2615, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:24.839773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:24.856533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1035/2094] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:24 2022] PUT /identity/v3/projects/f72843e62e2643178d716c409ffd22ac/users/c4fe8d0e1ec64d6f9b03def5173d2615/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:24.867252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f94b-d6e3-40fd-a04b-1fb1eb9d044b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:24.867542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f94b-d6e3-40fd-a04b-1fb1eb9d044b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:24.867841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f94b-d6e3-40fd-a04b-1fb1eb9d044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:24.898488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2877f94b-d6e3-40fd-a04b-1fb1eb9d044b None None] MFA Rules not processed for user `c4fe8d0e1ec64d6f9b03def5173d2615`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:24.899727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2877f94b-d6e3-40fd-a04b-1fb1eb9d044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:24.964248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1060/2095] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:24.968861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:25.034583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942f04-b32d-4278-8db7-8c429d16aa1a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:25.039952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:25.040643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.040953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.041229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.041945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:25.043827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:25.069861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1036/2096] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:24 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:25.073852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f94b-d6e3-40fd-a04b-1fb1eb9d044b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:25.085041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f94b-d6e3-40fd-a04b-1fb1eb9d044b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:25.141940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:25.142685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.142972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.143264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.143954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:25.145932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:25.166668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1061/2097] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:25 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:25.170667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:25.182118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184aa8d4-b7dd-484a-953a-65d2843f9f69 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:25.186886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:25.187545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.187826 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.188104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.188868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:25.190548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:25.201337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1037/2098] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:25.205756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:25.215990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97368e01-1a5d-424e-94eb-6951d6fe6937 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:25.221139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:25.224052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.224340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.224643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] PATH_INFO: `/v3/projects/15689e48d5b34cf7a0c7fee785e612ec/users/4e43f4884e9c4db7bcee5ee3bcb3d380/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.244965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15689e48d5b34cf7a0c7fee785e612ec, user_id=4e43f4884e9c4db7bcee5ee3bcb3d380, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:25.244965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:25.246238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:25.260240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1062/2099] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:25 2022] PUT /identity/v3/projects/15689e48d5b34cf7a0c7fee785e612ec/users/4e43f4884e9c4db7bcee5ee3bcb3d380/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:25.264146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:25.274524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05d5c-4ffd-4849-8940-8121cc7d3160 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:25.301642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:25.342970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09068727-312a-4221-bc26-b128f5078ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:25.343677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09068727-312a-4221-bc26-b128f5078ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.343993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09068727-312a-4221-bc26-b128f5078ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.344279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09068727-312a-4221-bc26-b128f5078ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.345050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09068727-312a-4221-bc26-b128f5078ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:25.346686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09068727-312a-4221-bc26-b128f5078ff1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:25.357449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1038/2100] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:25.361607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:25.371973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec595e2d-4edd-4242-9c27-c1c537c89839 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:25.376494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:25.377391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.377673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.377948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] PATH_INFO: `/v3/projects/15689e48d5b34cf7a0c7fee785e612ec/users/4e43f4884e9c4db7bcee5ee3bcb3d380/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.379949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15689e48d5b34cf7a0c7fee785e612ec, user_id=4e43f4884e9c4db7bcee5ee3bcb3d380, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:25.382616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:25.393166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1063/2101] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:25 2022] PUT /identity/v3/projects/15689e48d5b34cf7a0c7fee785e612ec/users/4e43f4884e9c4db7bcee5ee3bcb3d380/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:25.404774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe6f2-e839-4830-8552-8839dd425201 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.405069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe6f2-e839-4830-8552-8839dd425201 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.405352 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe6f2-e839-4830-8552-8839dd425201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.432384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9a6fe6f2-e839-4830-8552-8839dd425201 None None] MFA Rules not processed for user `4e43f4884e9c4db7bcee5ee3bcb3d380`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:25.433699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9a6fe6f2-e839-4830-8552-8839dd425201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:25.502149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1039/2102] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:25.513389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:25.515084 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:25.575066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d12385-1c9f-4e43-8271-4eade07093e3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:25.578588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bc786-2568-4a35-848b-5f371746266f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:25.579269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bc786-2568-4a35-848b-5f371746266f service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:25.579550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bc786-2568-4a35-848b-5f371746266f service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:25.579832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bc786-2568-4a35-848b-5f371746266f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:25.589987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bc786-2568-4a35-848b-5f371746266f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:25.591817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bc786-2568-4a35-848b-5f371746266f service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:25.603469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1064/2103] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:26.995843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe6f2-e839-4830-8552-8839dd425201 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:27.058544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe6f2-e839-4830-8552-8839dd425201 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:27.063067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:27.063934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:27.064217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:27.064500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] PATH_INFO: `/v3/users/c4fe8d0e1ec64d6f9b03def5173d2615` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:27.065724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4fe8d0e1ec64d6f9b03def5173d2615)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:27.068216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:27.112254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4fe8d0e1ec64d6f9b03def5173d2615'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.114395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:27.120724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4fe8d0e1ec64d6f9b03def5173d2615'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.127714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4fe8d0e1ec64d6f9b03def5173d2615'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.128350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4fe8d0e1ec64d6f9b03def5173d2615'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.129557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4fe8d0e1ec64d6f9b03def5173d2615'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.146400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1040/2104] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:26 2022] DELETE /identity/v3/users/c4fe8d0e1ec64d6f9b03def5173d2615 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:27.151037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:27.151524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:27.151619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:27.168861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:27.170044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:27.228386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1065/2105] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:27.237905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:27.239938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:27.296657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a9f2c-db69-419d-9222-4847d81aecae admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:27.300292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:27.301562 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:27.301842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:27.302116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:27.311284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:27.312730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:27.322030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1041/2106] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:27.623684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:27.625805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:27.667253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af277d-450e-4ff6-9fc5-d4dd0c965080 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:27.672473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:27.673425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:27.673711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:27.673991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] PATH_INFO: `/v3/projects/f72843e62e2643178d716c409ffd22ac` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:27.675292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f72843e62e2643178d716c409ffd22ac)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:27.677348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:27.732477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f72843e62e2643178d716c409ffd22ac', 'request_id': 'req-d99e8e67-0350-4b46-8189-36ded72abc95'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.741430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f72843e62e2643178d716c409ffd22ac', 'request_id': 'req-d99e8e67-0350-4b46-8189-36ded72abc95'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.750106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f72843e62e2643178d716c409ffd22ac', 'request_id': 'req-d99e8e67-0350-4b46-8189-36ded72abc95'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.763039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] The token cache is being invalidate because project f72843e62e2643178d716c409ffd22ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:27.763331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.764035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.766029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1066/2107] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:27 2022] DELETE /identity/v3/projects/f72843e62e2643178d716c409ffd22ac => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:27.770615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:27.780728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1606-af04-4e08-94ca-6a7f5b87db17 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:27.852236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:27.853408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:27.853784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:27.854132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] PATH_INFO: `/v3/users/4e43f4884e9c4db7bcee5ee3bcb3d380` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:27.855433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e43f4884e9c4db7bcee5ee3bcb3d380)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:27.857470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:27.912115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e43f4884e9c4db7bcee5ee3bcb3d380'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.922020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e43f4884e9c4db7bcee5ee3bcb3d380'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.929617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e43f4884e9c4db7bcee5ee3bcb3d380'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.930303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e43f4884e9c4db7bcee5ee3bcb3d380'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.930911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd15da20-3e83-4e6f-8520-ef9e9354c72e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e43f4884e9c4db7bcee5ee3bcb3d380'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:27.936168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:27.954049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1042/2108] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:27 2022] DELETE /identity/v3/users/4e43f4884e9c4db7bcee5ee3bcb3d380 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:28.029844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.031501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:28.053390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8e67-0350-4b46-8189-36ded72abc95 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.059788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:28.147003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.148124 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.148483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.148870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] PATH_INFO: `/v3/users/ef5d8eee69c549a9bb994b1046b9250d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.152790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5d8eee69c549a9bb994b1046b9250d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.154893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.218694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5d8eee69c549a9bb994b1046b9250d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.227277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5d8eee69c549a9bb994b1046b9250d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.235452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5d8eee69c549a9bb994b1046b9250d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.236771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5d8eee69c549a9bb994b1046b9250d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.237412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5d8eee69c549a9bb994b1046b9250d'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.258133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1067/2109] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:28 2022] DELETE /identity/v3/users/ef5d8eee69c549a9bb994b1046b9250d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:28.262667 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.263088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.263229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.281680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:28.283287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:28.317088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.318532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:28.365434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1043/2110] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:28.374884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.376277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:28.412930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b21e6-3747-4b14-b783-18229528c194 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.418600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.419067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.419351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.419630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] PATH_INFO: `/v3/projects/15689e48d5b34cf7a0c7fee785e612ec` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.421604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15689e48d5b34cf7a0c7fee785e612ec)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.423529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.440289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba78425-eec7-4648-9c58-58055e4fbb94 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.444869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b4508-bff7-4a2b-b1e4-855ee2c97d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.445690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b4508-bff7-4a2b-b1e4-855ee2c97d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.445978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b4508-bff7-4a2b-b1e4-855ee2c97d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.446255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b4508-bff7-4a2b-b1e4-855ee2c97d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.457726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b4508-bff7-4a2b-b1e4-855ee2c97d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.458963 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:28.459895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b4508-bff7-4a2b-b1e4-855ee2c97d97 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.470890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1044/2111] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:21:28.479896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15689e48d5b34cf7a0c7fee785e612ec', 'request_id': 'req-672bb9f2-1415-42c2-8066-0f7edfa69aa1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.488662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15689e48d5b34cf7a0c7fee785e612ec', 'request_id': 'req-672bb9f2-1415-42c2-8066-0f7edfa69aa1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.495020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15689e48d5b34cf7a0c7fee785e612ec', 'request_id': 'req-672bb9f2-1415-42c2-8066-0f7edfa69aa1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.509988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] The token cache is being invalidate because project 15689e48d5b34cf7a0c7fee785e612ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:28.510277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.510998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-672bb9f2-1415-42c2-8066-0f7edfa69aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.513055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1068/2112] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:28 2022] DELETE /identity/v3/projects/15689e48d5b34cf7a0c7fee785e612ec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:28.519621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1bf4-6538-4827-9e4f-720b9b16eeb7 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.519914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1bf4-6538-4827-9e4f-720b9b16eeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.520194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1bf4-6538-4827-9e4f-720b9b16eeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.542113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-367d1bf4-6538-4827-9e4f-720b9b16eeb7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:28.543483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-367d1bf4-6538-4827-9e4f-720b9b16eeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:28.619126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1045/2113] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:28.630613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea65cab-e542-4b84-90d1-c430ad7473a4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.630909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea65cab-e542-4b84-90d1-c430ad7473a4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.631188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea65cab-e542-4b84-90d1-c430ad7473a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.652453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2ea65cab-e542-4b84-90d1-c430ad7473a4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:28.653708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2ea65cab-e542-4b84-90d1-c430ad7473a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:28.661956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1069/2114] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:28.667002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1bf4-6538-4827-9e4f-720b9b16eeb7 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.680881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1bf4-6538-4827-9e4f-720b9b16eeb7 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.688136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.688136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.688136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.688136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.689061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:28.689375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.692811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.710016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1046/2115] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:21:28.714598 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea65cab-e542-4b84-90d1-c430ad7473a4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.725345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea65cab-e542-4b84-90d1-c430ad7473a4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.731909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.732814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.733168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.733526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.734516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.736521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.773536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1070/2116] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:28 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:28.778644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.785537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.788027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:28.795527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e7719-173d-4ee9-b542-59ea3e7dd03e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.824425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:28.871956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.872650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.872939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.873219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.874023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.874926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd8c8d-561f-4751-9452-7eb7d147edfd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.875978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.880238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.881131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.881414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.881716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] PATH_INFO: `/v3/projects/148861fd67cb4d0380fce5b62a8a7318` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.882905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=148861fd67cb4d0380fce5b62a8a7318)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.884967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.899694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1047/2117] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:28 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:28.900840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.911207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdf81b-e9ae-49c7-bb45-0ebd9488cd36 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.916916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:28.917534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.917843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.918196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.919036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:28.920887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:28.932225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1048/2118] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:28.936094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:28.937119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148861fd67cb4d0380fce5b62a8a7318', 'request_id': 'req-75c44726-cc36-4b3e-9f33-3e241f076a64'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.945983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148861fd67cb4d0380fce5b62a8a7318', 'request_id': 'req-75c44726-cc36-4b3e-9f33-3e241f076a64'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.946845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfa416-5e3c-457f-8679-33bfe5cb1f08 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:28.953047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148861fd67cb4d0380fce5b62a8a7318', 'request_id': 'req-75c44726-cc36-4b3e-9f33-3e241f076a64'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.965096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] The token cache is being invalidate because project 148861fd67cb4d0380fce5b62a8a7318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:28.965392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.966095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75c44726-cc36-4b3e-9f33-3e241f076a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:28.967712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1071/2119] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:28 2022] DELETE /identity/v3/projects/148861fd67cb4d0380fce5b62a8a7318 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:28.972847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba9121-03e3-4c51-8031-9ba3c84f1804 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:28.973325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba9121-03e3-4c51-8031-9ba3c84f1804 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:28.973466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba9121-03e3-4c51-8031-9ba3c84f1804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:28.992648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3cba9121-03e3-4c51-8031-9ba3c84f1804 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:28.993875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3cba9121-03e3-4c51-8031-9ba3c84f1804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:29.007957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:29.009983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.010845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.011032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.011365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.012028 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.013762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.029036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1049/2120] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:29.029969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.033066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:29.049357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2addf9-29de-4b55-832e-db01685afad7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.055104 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7515661-882f-40c0-8be1-1acf8ff8366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.055998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515661-882f-40c0-8be1-1acf8ff8366e admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.056283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515661-882f-40c0-8be1-1acf8ff8366e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.056872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515661-882f-40c0-8be1-1acf8ff8366e admin admin] PATH_INFO: `/v3/projects/c1603266b43e44db905af1701c0b9348/users/0b9f08c3fe0b4a3eb03301cef4d49fcc/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.072851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1072/2121] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:29.073433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7515661-882f-40c0-8be1-1acf8ff8366e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1603266b43e44db905af1701c0b9348, user_id=0b9f08c3fe0b4a3eb03301cef4d49fcc, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.076217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7515661-882f-40c0-8be1-1acf8ff8366e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.081161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75272-7597-410e-8ccd-0c6f0a9681f3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.081461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75272-7597-410e-8ccd-0c6f0a9681f3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.082360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75272-7597-410e-8ccd-0c6f0a9681f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.100606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1050/2122] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:29 2022] PUT /identity/v3/projects/c1603266b43e44db905af1701c0b9348/users/0b9f08c3fe0b4a3eb03301cef4d49fcc/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:29.101215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-98d75272-7597-410e-8ccd-0c6f0a9681f3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:29.102420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-98d75272-7597-410e-8ccd-0c6f0a9681f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:29.105071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c4cd-27b0-4ccb-88eb-bca1426a4511 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.105324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c4cd-27b0-4ccb-88eb-bca1426a4511 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.105604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c4cd-27b0-4ccb-88eb-bca1426a4511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.134284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3438c4cd-27b0-4ccb-88eb-bca1426a4511 None None] MFA Rules not processed for user `0b9f08c3fe0b4a3eb03301cef4d49fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:29.135494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3438c4cd-27b0-4ccb-88eb-bca1426a4511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:29.172379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1073/2123] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:29.174058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d75272-7597-410e-8ccd-0c6f0a9681f3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.183445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d75272-7597-410e-8ccd-0c6f0a9681f3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.188648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.189572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.189999 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.190381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.191277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:29.191594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.193270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.202855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:29.206209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1074/2124] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:21:29.211607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.216730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1051/2125] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:29.217790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438c4cd-27b0-4ccb-88eb-bca1426a4511 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.224892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32961c8-efd9-44d7-bad2-386e9ea020e5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.229884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.230799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.231079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438c4cd-27b0-4ccb-88eb-bca1426a4511 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.231204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.231549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.232444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.235200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.235915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.236577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.236889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.237167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.237855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.239756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.263158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1075/2126] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:29.266546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1052/2127] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:29.267487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.271090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.335068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64455ba-8f55-42ab-b29c-db2d61bf2ae1 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.336275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cac74c-a4f9-4fee-9018-1b51f6ea1f7e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.339868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.340538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.340850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.340971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.341127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.341513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.341816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.341909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.342115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.342796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.343727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.344752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.364665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1053/2128] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:29.365492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1076/2129] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:29.368727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.369104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.378333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09868a9-40d7-48c7-829e-bfc2df1d5e0f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.378828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2570f4-320f-4650-affc-a2038dc85054 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.383016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.383484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.383644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.383922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.384126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.384241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.384423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.384740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.385047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.385455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.386857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.387022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.397405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1054/2130] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:29.400513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1077/2131] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:29.401603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.401745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.406701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:29.412167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb56b-da43-4bf1-8fd7-6e5f7b15bbc1 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.414139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2adcdd-283a-4145-83d6-ca825b74e021 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.416918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.417562 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.417994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.418277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.418999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.419101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.419950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.420234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.420516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] PATH_INFO: `/v3/projects/00d9fb08064349d4a63f796c84e6ae15/users/a8455be82436434a9c60bf6867e9b214/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.420639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.434623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1055/2132] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:29.435513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.436724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d9fb08064349d4a63f796c84e6ae15, user_id=a8455be82436434a9c60bf6867e9b214, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.439407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.445530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7c2cb-427d-45ba-959a-1423a8314ff3 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.450671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866a82f-295b-408a-8450-280c2f520648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.451536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866a82f-295b-408a-8450-280c2f520648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.451818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866a82f-295b-408a-8450-280c2f520648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.452093 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866a82f-295b-408a-8450-280c2f520648 admin admin] PATH_INFO: `/v3/projects/88adc93662f14d8b9e133856e88ea464/users/d062912dfeb34d31a396b7b076e14aac/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.453679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1078/2133] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:29 2022] PUT /identity/v3/projects/00d9fb08064349d4a63f796c84e6ae15/users/a8455be82436434a9c60bf6867e9b214/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:29.458040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.467190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866a82f-295b-408a-8450-280c2f520648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88adc93662f14d8b9e133856e88ea464, user_id=d062912dfeb34d31a396b7b076e14aac, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.469910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866a82f-295b-408a-8450-280c2f520648 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.482932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1056/2134] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:29 2022] PUT /identity/v3/projects/88adc93662f14d8b9e133856e88ea464/users/d062912dfeb34d31a396b7b076e14aac/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:29.494949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12363480-9520-4e4d-b678-4f47c3ba5f14 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.495240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12363480-9520-4e4d-b678-4f47c3ba5f14 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.495519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12363480-9520-4e4d-b678-4f47c3ba5f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.522664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d2849-ce04-4e62-85ba-9510a47d2a40 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.524161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-12363480-9520-4e4d-b678-4f47c3ba5f14 None None] MFA Rules not processed for user `d062912dfeb34d31a396b7b076e14aac`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:29.525373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-12363480-9520-4e4d-b678-4f47c3ba5f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:29.578968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.579792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.580606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.580891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.581612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.584107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.589506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1057/2135] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:29.593659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12363480-9520-4e4d-b678-4f47c3ba5f14 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.594632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1079/2136] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:29.598515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.604218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12363480-9520-4e4d-b678-4f47c3ba5f14 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.608214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25bb1a-3aa4-4433-872c-c921fa8e5782 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.608956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.609603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.609885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.610188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.610880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.612789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.612989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f035a-6bce-4371-b537-e11d7888af9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.613855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f035a-6bce-4371-b537-e11d7888af9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.614142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f035a-6bce-4371-b537-e11d7888af9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.614419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f035a-6bce-4371-b537-e11d7888af9f admin admin] PATH_INFO: `/v3/projects/00d9fb08064349d4a63f796c84e6ae15/users/a8455be82436434a9c60bf6867e9b214/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.616374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f035a-6bce-4371-b537-e11d7888af9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d9fb08064349d4a63f796c84e6ae15, user_id=a8455be82436434a9c60bf6867e9b214, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.616994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:29.619112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f035a-6bce-4371-b537-e11d7888af9f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.630793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1080/2137] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:29 2022] PUT /identity/v3/projects/00d9fb08064349d4a63f796c84e6ae15/users/a8455be82436434a9c60bf6867e9b214/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:29.641225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.641518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.641793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.642648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1058/2138] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:29.646658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.672126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] MFA Rules not processed for user `a8455be82436434a9c60bf6867e9b214`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:29.673514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:29.713094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fca83c-c877-4897-9bf4-791763b8f413 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.718931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.719647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.719914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.720193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.720924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.722959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.733362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:29.745651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1059/2139] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:29.745813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1081/2140] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:29.746699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.756221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.757340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234f6fe-64b6-4a4c-929a-f9c5a802c64a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.757627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:29.761961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.762734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.763019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.763297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.764025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.765640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.776189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1060/2141] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:29.780649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.792801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dacc3-407d-4145-bcea-a0cf5dad7d86 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.797490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.798357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.798639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.798915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] PATH_INFO: `/v3/projects/7609c778513e45618ec1f43fec9170ff/users/edff52be6c264b6dab374d24b73a4a6a/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.814532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7609c778513e45618ec1f43fec9170ff, user_id=edff52be6c264b6dab374d24b73a4a6a, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.814900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e790d2a-4129-46df-845b-8a5886879133 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.817802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.819300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.819943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.820228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.820508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.831594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.833789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.836262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1061/2142] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:29 2022] PUT /identity/v3/projects/7609c778513e45618ec1f43fec9170ff/users/edff52be6c264b6dab374d24b73a4a6a/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:29.840211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.844866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1082/2143] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:29.919720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdde1b3-720a-42c7-bcd8-2805d31ff19e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.926018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d35d31-5fff-477b-8625-27cd4d21e2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.926624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d35d31-5fff-477b-8625-27cd4d21e2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.926891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d35d31-5fff-477b-8625-27cd4d21e2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.927423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d35d31-5fff-477b-8625-27cd4d21e2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.927888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d35d31-5fff-477b-8625-27cd4d21e2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.929808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d35d31-5fff-477b-8625-27cd4d21e2a0 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.942261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1062/2144] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:29.946477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:29.960616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523e96-95d5-49f1-8456-c9e0d92f6519 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:29.965623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:29.966547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.966831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.967108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] PATH_INFO: `/v3/projects/7609c778513e45618ec1f43fec9170ff/users/edff52be6c264b6dab374d24b73a4a6a/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:29.969703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7609c778513e45618ec1f43fec9170ff, user_id=edff52be6c264b6dab374d24b73a4a6a, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:29.973746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:29.988796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1083/2145] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:29 2022] PUT /identity/v3/projects/7609c778513e45618ec1f43fec9170ff/users/edff52be6c264b6dab374d24b73a4a6a/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:29.997149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d256da-68c9-448d-9310-8475329f3d79 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:29.997445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d256da-68c9-448d-9310-8475329f3d79 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:29.997738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d256da-68c9-448d-9310-8475329f3d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:30.033678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-22d256da-68c9-448d-9310-8475329f3d79 None None] MFA Rules not processed for user `edff52be6c264b6dab374d24b73a4a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:30.035174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-22d256da-68c9-448d-9310-8475329f3d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:30.069838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:30.119445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1063/2146] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:30.132248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:30.194868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cbf36-7c55-49d2-9faa-5d24ef42d39c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:30.198616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf68ca0-20ef-496c-9642-e6302851a505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:30.199268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf68ca0-20ef-496c-9642-e6302851a505 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:30.199550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf68ca0-20ef-496c-9642-e6302851a505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:30.199824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf68ca0-20ef-496c-9642-e6302851a505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:30.267747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf68ca0-20ef-496c-9642-e6302851a505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:30.269248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf68ca0-20ef-496c-9642-e6302851a505 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:30.278778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1084/2147] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:30.441596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d256da-68c9-448d-9310-8475329f3d79 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:30.514025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d256da-68c9-448d-9310-8475329f3d79 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:30.519848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:30.520837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:30.521191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:30.521553 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] PATH_INFO: `/v3/users/0b9f08c3fe0b4a3eb03301cef4d49fcc` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:30.522796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9f08c3fe0b4a3eb03301cef4d49fcc)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:30.524827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:30.584453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9f08c3fe0b4a3eb03301cef4d49fcc'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:30.593456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9f08c3fe0b4a3eb03301cef4d49fcc'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:30.602003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9f08c3fe0b4a3eb03301cef4d49fcc'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:30.602715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9f08c3fe0b4a3eb03301cef4d49fcc'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:30.603335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9f08c3fe0b4a3eb03301cef4d49fcc'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:30.621350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1064/2148] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:30 2022] DELETE /identity/v3/users/0b9f08c3fe0b4a3eb03301cef4d49fcc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:30.625987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:30.626280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:30.626563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:30.645839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:30.647047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:30.680208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:30.708788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1085/2149] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:30.718336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:30.719727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:30.765494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:30.778819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39917d6-e923-4df8-a8a8-0f6267c66f55 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:30.782464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:30.783105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:30.783389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:30.783670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:30.793292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:30.795022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:30.804652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1065/2150] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:31.018382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:31.024791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:31.033494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0fd33-7f1f-4dc8-88a7-3f0f399f2801 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:31.038298 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:31.039578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.039578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.039811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] PATH_INFO: `/v3/users/648868f0f8674fe294df4b0ce2e9bdf6` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.041030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648868f0f8674fe294df4b0ce2e9bdf6)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:31.042951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:31.080777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:31.082820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:31.099319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c47424-c1c9-4c77-9a6e-12c01b20130f service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:31.101640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648868f0f8674fe294df4b0ce2e9bdf6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.105062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:31.106124 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.106409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.106684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] PATH_INFO: `/v3/projects/c1603266b43e44db905af1701c0b9348` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.108538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1603266b43e44db905af1701c0b9348)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:31.110901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:31.112185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648868f0f8674fe294df4b0ce2e9bdf6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.119652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648868f0f8674fe294df4b0ce2e9bdf6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.120379 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648868f0f8674fe294df4b0ce2e9bdf6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.121017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-85549f63-28aa-4f65-8171-58c5556f3f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648868f0f8674fe294df4b0ce2e9bdf6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.139729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1086/2151] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:31 2022] DELETE /identity/v3/users/648868f0f8674fe294df4b0ce2e9bdf6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:31.144015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.144898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.145180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.159800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1603266b43e44db905af1701c0b9348', 'request_id': 'req-e95005a6-cb62-4fa0-b97b-c52828fc4652'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.162489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:31.163686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:31.168511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1603266b43e44db905af1701c0b9348', 'request_id': 'req-e95005a6-cb62-4fa0-b97b-c52828fc4652'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.174639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1603266b43e44db905af1701c0b9348', 'request_id': 'req-e95005a6-cb62-4fa0-b97b-c52828fc4652'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.185812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] The token cache is being invalidate because project c1603266b43e44db905af1701c0b9348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:31.186283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.186928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.188462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1066/2152] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:31 2022] DELETE /identity/v3/projects/c1603266b43e44db905af1701c0b9348 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:31.192986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:31.194354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:31.226583 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1087/2153] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:31.236114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:31.237462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:31.262958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95005a6-cb62-4fa0-b97b-c52828fc4652 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:31.267642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:31.268449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.268754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.269031 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] PATH_INFO: `/v3/users/a8455be82436434a9c60bf6867e9b214` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.270794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8455be82436434a9c60bf6867e9b214)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:31.272727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:31.293797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf177e4-b578-4363-9106-25237add3f51 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:31.297773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:31.298414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.298711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.298969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.306077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:31.310565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:31.313102 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:31.322511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8455be82436434a9c60bf6867e9b214'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.326937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1088/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:21:31.330569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8455be82436434a9c60bf6867e9b214'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.338617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8455be82436434a9c60bf6867e9b214'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.339303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8455be82436434a9c60bf6867e9b214'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.339945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8455be82436434a9c60bf6867e9b214'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.345693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:31.361898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1067/2155] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:31 2022] DELETE /identity/v3/users/a8455be82436434a9c60bf6867e9b214 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:31.782027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:31.783541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:31.804475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf945cd-d902-4729-af97-06fda06e8e17 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:31.849437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:31.850921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:31.892032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:31.893178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.893504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.894091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] PATH_INFO: `/v3/projects/1381dac7b3a64ee5a3ceeed2f1406a12` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.896138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1381dac7b3a64ee5a3ceeed2f1406a12)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:31.898108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:31.934095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc6dc0-c6bb-4d22-8e0d-07abb882ca3a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:31.939173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:31.940091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.940372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.940827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] PATH_INFO: `/v3/projects/00d9fb08064349d4a63f796c84e6ae15` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.942598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d9fb08064349d4a63f796c84e6ae15)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:31.944530 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:31.951204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1381dac7b3a64ee5a3ceeed2f1406a12', 'request_id': 'req-d0e89b02-0181-4b65-92f8-9b996e46ab3b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.959835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1381dac7b3a64ee5a3ceeed2f1406a12', 'request_id': 'req-d0e89b02-0181-4b65-92f8-9b996e46ab3b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.965722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1381dac7b3a64ee5a3ceeed2f1406a12', 'request_id': 'req-d0e89b02-0181-4b65-92f8-9b996e46ab3b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.977920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] The token cache is being invalidate because project 1381dac7b3a64ee5a3ceeed2f1406a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:31.978278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.978869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d0e89b02-0181-4b65-92f8-9b996e46ab3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:31.980585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1089/2156] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:31 2022] DELETE /identity/v3/projects/1381dac7b3a64ee5a3ceeed2f1406a12 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:31.986193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1da8e-0fa7-4728-8ce4-cadce841f906 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:31.987132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1da8e-0fa7-4728-8ce4-cadce841f906 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:31.987790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1da8e-0fa7-4728-8ce4-cadce841f906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:31.995318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d9fb08064349d4a63f796c84e6ae15', 'request_id': 'req-7c2c130b-2625-4eaf-9794-28789c25d4e9'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.003396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d9fb08064349d4a63f796c84e6ae15', 'request_id': 'req-7c2c130b-2625-4eaf-9794-28789c25d4e9'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.010949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f2f1da8e-0fa7-4728-8ce4-cadce841f906 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:32.011119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d9fb08064349d4a63f796c84e6ae15', 'request_id': 'req-7c2c130b-2625-4eaf-9794-28789c25d4e9'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.012151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f2f1da8e-0fa7-4728-8ce4-cadce841f906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.024724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] The token cache is being invalidate because project 00d9fb08064349d4a63f796c84e6ae15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:32.024995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.025610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-7c2c130b-2625-4eaf-9794-28789c25d4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.027194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1068/2157] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:31 2022] DELETE /identity/v3/projects/00d9fb08064349d4a63f796c84e6ae15 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:32.031761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db1b5a-f3b3-4eff-92fa-a6eb2aaab0d7 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.032051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db1b5a-f3b3-4eff-92fa-a6eb2aaab0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.032331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db1b5a-f3b3-4eff-92fa-a6eb2aaab0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.050102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a3db1b5a-f3b3-4eff-92fa-a6eb2aaab0d7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:32.051351 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a3db1b5a-f3b3-4eff-92fa-a6eb2aaab0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.073978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1090/2158] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:32.082933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba519b-926b-4d5c-bb44-aa08e4f2e855 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.083230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba519b-926b-4d5c-bb44-aa08e4f2e855 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.083514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba519b-926b-4d5c-bb44-aa08e4f2e855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.101515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fdba519b-926b-4d5c-bb44-aa08e4f2e855 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:32.102717 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fdba519b-926b-4d5c-bb44-aa08e4f2e855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.114547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1091/2159] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:32.114770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1069/2160] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:32.115552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba519b-926b-4d5c-bb44-aa08e4f2e855 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.119401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55eba7-22ee-4e29-9b1d-2a50e36f1f1b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.119745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55eba7-22ee-4e29-9b1d-2a50e36f1f1b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.120023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55eba7-22ee-4e29-9b1d-2a50e36f1f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.125033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba519b-926b-4d5c-bb44-aa08e4f2e855 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.132103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.133702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.134144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.134708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.135912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:32.136366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.138179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.140151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-4b55eba7-22ee-4e29-9b1d-2a50e36f1f1b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:32.141386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4b55eba7-22ee-4e29-9b1d-2a50e36f1f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.151064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1070/2161] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:32.155023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1092/2162] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:21:32.155287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55eba7-22ee-4e29-9b1d-2a50e36f1f1b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.159029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.164665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55eba7-22ee-4e29-9b1d-2a50e36f1f1b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.165216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:32.169563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa8c60-9fee-4691-bf68-bf40b01e8515 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.170022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.170679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.171078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.171381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.172248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:32.172611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.174281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.176606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.177596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.179142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.179504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.180506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.182554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:32.182754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.191978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1071/2163] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:21:32.194513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.205268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041e44f-9b5d-41d5-b1cd-1b81d584543b admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.209943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.210587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.210884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.214326 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1093/2164] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:32.214468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.214468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.214468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.215206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.242949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1072/2165] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:32.243919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.279858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ebd97-9647-42a6-9860-42cb37778058 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.307341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022acc8-e727-448e-8b5e-bbe3faebe5c9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.312678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.314122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.314413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.314695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.315391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.317942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.337665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.338030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1073/2166] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:32.341070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.341373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.341724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.341963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.342599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.344611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.351520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc221-4155-4917-8ba6-00b27510d2c7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.357084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.357825 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.358116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.358402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.359181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.360818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.370396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1094/2167] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:32.371464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.371638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1074/2168] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:32.375291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.381655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e811-f7a5-4deb-9329-7b6c1ff56572 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.385454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5db03-0c38-42a3-98b6-a152a20f9640 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.386556 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.387205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.387487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.387764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.388482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.390281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.390481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.390921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.391849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.392140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.392900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.394501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.400827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1095/2169] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:32.404427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.405800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1075/2170] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:32.409527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.413941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84ad00-2026-4d84-8348-719dae2ea00a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.418905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf76f4-1dd3-4496-91e6-e5c48e1725d8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.419022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.419585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.419869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.420145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.420913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.422484 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.423701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fa65c-cf1f-4718-afe4-09af1f89a219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.424576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fa65c-cf1f-4718-afe4-09af1f89a219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.424874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fa65c-cf1f-4718-afe4-09af1f89a219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.425153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fa65c-cf1f-4718-afe4-09af1f89a219 admin admin] PATH_INFO: `/v3/projects/9c4081d5c6b14a56aef0c9cb2271b2ca/users/c1e9d82266e74bd5a2c02ab1c9c1bcf4/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.433573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1096/2171] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:32.437454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.440481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fa65c-cf1f-4718-afe4-09af1f89a219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4081d5c6b14a56aef0c9cb2271b2ca, user_id=c1e9d82266e74bd5a2c02ab1c9c1bcf4, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.443745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fa65c-cf1f-4718-afe4-09af1f89a219 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.447309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971734f0-9327-46ae-a467-7f35ba92d2bf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.452933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96957269-90d2-443c-912a-c4d10c2ad1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.453799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96957269-90d2-443c-912a-c4d10c2ad1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.454080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96957269-90d2-443c-912a-c4d10c2ad1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.454355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96957269-90d2-443c-912a-c4d10c2ad1b7 admin admin] PATH_INFO: `/v3/projects/22e65c0cc9524c54881d604ed1291a01/users/dfc2fed192eb456d87b711d0d44e107f/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.456286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1076/2172] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:32 2022] PUT /identity/v3/projects/9c4081d5c6b14a56aef0c9cb2271b2ca/users/c1e9d82266e74bd5a2c02ab1c9c1bcf4/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:32.467486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.467778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.468061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.469747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96957269-90d2-443c-912a-c4d10c2ad1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e65c0cc9524c54881d604ed1291a01, user_id=dfc2fed192eb456d87b711d0d44e107f, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.472957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96957269-90d2-443c-912a-c4d10c2ad1b7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.485646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1097/2173] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:32 2022] PUT /identity/v3/projects/22e65c0cc9524c54881d604ed1291a01/users/dfc2fed192eb456d87b711d0d44e107f/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:32.491808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15006b99-314d-4660-adff-4d560b697c6e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.492096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15006b99-314d-4660-adff-4d560b697c6e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.492371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15006b99-314d-4660-adff-4d560b697c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.497278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] MFA Rules not processed for user `c1e9d82266e74bd5a2c02ab1c9c1bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:32.498476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.508593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-15006b99-314d-4660-adff-4d560b697c6e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:32.509947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-15006b99-314d-4660-adff-4d560b697c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.541474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:32.563491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1077/2174] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:32.573856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1098/2175] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:32.574637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.576024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.583078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15006b99-314d-4660-adff-4d560b697c6e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.584471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-15006b99-314d-4660-adff-4d560b697c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.586543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:32.635266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560f8a-80ff-4296-923d-c7f7e5e97a79 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.638896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.639545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.639827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.640100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.641854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15006b99-314d-4660-adff-4d560b697c6e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.645627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee02dc4-cc4f-4bdd-b88f-dd6b5023bcc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.646283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02dc4-cc4f-4bdd-b88f-dd6b5023bcc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.646566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02dc4-cc4f-4bdd-b88f-dd6b5023bcc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.646835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02dc4-cc4f-4bdd-b88f-dd6b5023bcc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.648635 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.650066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.655951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee02dc4-cc4f-4bdd-b88f-dd6b5023bcc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.657382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee02dc4-cc4f-4bdd-b88f-dd6b5023bcc5 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.659271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1078/2176] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:21:32.667615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1099/2177] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:21:32.680531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.691085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb5672-5b25-4e46-8b7f-b2bd4b9a9fe0 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:32.696355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:32.697240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.697519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.697789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] PATH_INFO: `/v3/users/c1e9d82266e74bd5a2c02ab1c9c1bcf4` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.698984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e9d82266e74bd5a2c02ab1c9c1bcf4)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:32.700937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:32.758330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e9d82266e74bd5a2c02ab1c9c1bcf4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.767222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e9d82266e74bd5a2c02ab1c9c1bcf4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.776396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e9d82266e74bd5a2c02ab1c9c1bcf4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.777159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e9d82266e74bd5a2c02ab1c9c1bcf4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.777771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e9d82266e74bd5a2c02ab1c9c1bcf4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:32.797812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1079/2178] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:32 2022] DELETE /identity/v3/users/c1e9d82266e74bd5a2c02ab1c9c1bcf4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:32.802721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:32.803020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:32.803464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:32.822396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:32.823273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.898510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.900583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:32.924877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1100/2179] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:21:32.933492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:32.935092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:33.017174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e140-95b4-4bbc-a5be-72ab702c9032 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:33.026103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:33.027115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.027471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.027819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] PATH_INFO: `/v3/users/d062912dfeb34d31a396b7b076e14aac` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.029138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d062912dfeb34d31a396b7b076e14aac)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:33.031206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:33.039810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a530fc-c288-4c51-a015-9ad0508602b7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:33.043876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bffeb-4db2-421a-b8d5-55389227a783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:33.045238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bffeb-4db2-421a-b8d5-55389227a783 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.045535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bffeb-4db2-421a-b8d5-55389227a783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.045820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bffeb-4db2-421a-b8d5-55389227a783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.058680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bffeb-4db2-421a-b8d5-55389227a783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:33.061915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bffeb-4db2-421a-b8d5-55389227a783 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:33.067607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:33.078549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1101/2180] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:21:33.098327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd062912dfeb34d31a396b7b076e14aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.107110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd062912dfeb34d31a396b7b076e14aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.114485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd062912dfeb34d31a396b7b076e14aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.115151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd062912dfeb34d31a396b7b076e14aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.115753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd062912dfeb34d31a396b7b076e14aac'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.134447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1080/2181] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:32 2022] DELETE /identity/v3/users/d062912dfeb34d31a396b7b076e14aac => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:33.139232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.139526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.139811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.158440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:33.159676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:33.174647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:33.224918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1102/2182] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:33.234725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:33.236152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:33.295660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893af53-48e0-4ab9-9cae-5646a4780dd3 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:33.299318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a829a20-5194-4f89-857c-1e20e13d53a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:33.299964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a829a20-5194-4f89-857c-1e20e13d53a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.300251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a829a20-5194-4f89-857c-1e20e13d53a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.300720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a829a20-5194-4f89-857c-1e20e13d53a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.310226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a829a20-5194-4f89-857c-1e20e13d53a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:33.311877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a829a20-5194-4f89-857c-1e20e13d53a6 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:33.326192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1081/2183] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:21:33.611979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:33.615353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:33.628000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55605e84-f51c-48a4-b748-70b55eb31542 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:33.633758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:33.634987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.635397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.635684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] PATH_INFO: `/v3/projects/9c4081d5c6b14a56aef0c9cb2271b2ca` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.637011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4081d5c6b14a56aef0c9cb2271b2ca)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:33.639010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:33.704945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4081d5c6b14a56aef0c9cb2271b2ca', 'request_id': 'req-49c1d79a-7fdc-4d36-8191-a0cc376d569f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.714961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4081d5c6b14a56aef0c9cb2271b2ca', 'request_id': 'req-49c1d79a-7fdc-4d36-8191-a0cc376d569f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.721367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4081d5c6b14a56aef0c9cb2271b2ca', 'request_id': 'req-49c1d79a-7fdc-4d36-8191-a0cc376d569f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.736108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] The token cache is being invalidate because project 9c4081d5c6b14a56aef0c9cb2271b2ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:33.736380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.737545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49c1d79a-7fdc-4d36-8191-a0cc376d569f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:33.739936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1103/2184] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:33 2022] DELETE /identity/v3/projects/9c4081d5c6b14a56aef0c9cb2271b2ca => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:33.744882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b185-548d-47ab-b876-4a5c4dc092da None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.745171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b185-548d-47ab-b876-4a5c4dc092da None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.745448 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b185-548d-47ab-b876-4a5c4dc092da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.765508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-0b18b185-548d-47ab-b876-4a5c4dc092da None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:33.766734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0b18b185-548d-47ab-b876-4a5c4dc092da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:33.852749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1082/2185] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:33.860350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.860998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.861579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.886274 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:33.888192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:33.903661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1104/2186] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:33.908326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18b185-548d-47ab-b876-4a5c4dc092da None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:33.935451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18b185-548d-47ab-b876-4a5c4dc092da None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:33.950520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:33.951470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:33.952289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:33.952809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:33.953646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:33.954065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:33.955915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:33.964778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:33.966517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:33.973178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1083/2187] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:21:33.978688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:33.994318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec46de-43c6-496d-82d5-8ca03c7a46fb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.002413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9634322-5267-4d27-8b1f-2a78952ca6ab admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.006193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.007552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.007843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.008120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] PATH_INFO: `/v3/projects/88adc93662f14d8b9e133856e88ea464` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.012639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=88adc93662f14d8b9e133856e88ea464)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.015269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.019307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.019936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.020216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.021130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.021841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.025270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.080737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1084/2188] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:33 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:34.085004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:34.116724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:34.129900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88adc93662f14d8b9e133856e88ea464', 'request_id': 'req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.142443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88adc93662f14d8b9e133856e88ea464', 'request_id': 'req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.154721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88adc93662f14d8b9e133856e88ea464', 'request_id': 'req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.182185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] The token cache is being invalidate because project 88adc93662f14d8b9e133856e88ea464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:34.183507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087005d-14c8-4a1d-b99c-b7cdc0f388b4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.184629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.185618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.187255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1105/2189] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:33 2022] DELETE /identity/v3/projects/88adc93662f14d8b9e133856e88ea464 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:34.193665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:34.196208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:34.252731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:34.274601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.276007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.276390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.276796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.278113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.281153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.309080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc37364-d0fe-4bb4-a0ea-eef0f00847bc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.319433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.323922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1085/2190] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:34 2022] POST /identity/v3/users => generated 288 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:34.325366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.326214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.328247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] PATH_INFO: `/v3/users/edff52be6c264b6dab374d24b73a4a6a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.330502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] RBAC: Authorizing `identity:delete_user(user_id=edff52be6c264b6dab374d24b73a4a6a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.334535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.335469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:34.337347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:34.351082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55380a2-e65d-4b94-bde2-424ac6244a3c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.357188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.357869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.358157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.358468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.359466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.360934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.375342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1086/2191] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:34.379989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:34.391579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4914-2d06-4727-bade-ee3e4b2c5242 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.400398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.402727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.403024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.403301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.404089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.405748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.413646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edff52be6c264b6dab374d24b73a4a6a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.419409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1087/2192] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:34.423933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:34.426111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edff52be6c264b6dab374d24b73a4a6a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.434258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edff52be6c264b6dab374d24b73a4a6a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.434972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edff52be6c264b6dab374d24b73a4a6a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.435760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-43689ad5-ef6b-4dbd-8160-fce9921cbfff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edff52be6c264b6dab374d24b73a4a6a'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:34.436921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbf9be-c2fe-4c61-9b83-fb94ba026fcb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.441974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.442991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.443375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.443785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] PATH_INFO: `/v3/projects/1fcfd950d921449f98b14d6c63bf8793/users/3431c8ec634140039928dbeba141bc01/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.454861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1106/2193] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:21:34 2022] DELETE /identity/v3/users/edff52be6c264b6dab374d24b73a4a6a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:34.469616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fcfd950d921449f98b14d6c63bf8793, user_id=3431c8ec634140039928dbeba141bc01, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.473605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.508878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.509273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.509652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.511235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1088/2194] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:34 2022] PUT /identity/v3/projects/1fcfd950d921449f98b14d6c63bf8793/users/3431c8ec634140039928dbeba141bc01/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:21:34.567214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] MFA Rules not processed for user `3431c8ec634140039928dbeba141bc01`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:34.569060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:34.683744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1107/2195] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:34.698852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:34.701763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:34.782246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4fb34-48d7-467c-8dc8-983061c9fcfb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.786238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da0b48-46e4-4309-8cea-ddb36c3b58f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.786897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da0b48-46e4-4309-8cea-ddb36c3b58f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.787182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da0b48-46e4-4309-8cea-ddb36c3b58f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.787455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da0b48-46e4-4309-8cea-ddb36c3b58f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.797816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da0b48-46e4-4309-8cea-ddb36c3b58f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.799357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da0b48-46e4-4309-8cea-ddb36c3b58f6 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:34.815782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1089/2196] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:34.850386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:34.851937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:34.893193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:34.941898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f94d8-dcaa-48c0-8dcc-20d9be987f23 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:34.947118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:34.949851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:34.950221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:34.950573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] PATH_INFO: `/v3/projects/7609c778513e45618ec1f43fec9170ff` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:34.952548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7609c778513e45618ec1f43fec9170ff)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:34.955177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.026813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7609c778513e45618ec1f43fec9170ff', 'request_id': 'req-ae95981d-2a7d-46c3-8620-83c68c981ea4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:35.039215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7609c778513e45618ec1f43fec9170ff', 'request_id': 'req-ae95981d-2a7d-46c3-8620-83c68c981ea4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:35.047917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7609c778513e45618ec1f43fec9170ff', 'request_id': 'req-ae95981d-2a7d-46c3-8620-83c68c981ea4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:35.062034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] The token cache is being invalidate because project 7609c778513e45618ec1f43fec9170ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:21:35.062432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:35.063168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-ae95981d-2a7d-46c3-8620-83c68c981ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:21:35.064877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1108/2197] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:21:34 2022] DELETE /identity/v3/projects/7609c778513e45618ec1f43fec9170ff => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:35.070422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34337cb5-4e3b-48b7-b372-47993499a293 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.070932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34337cb5-4e3b-48b7-b372-47993499a293 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.070932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34337cb5-4e3b-48b7-b372-47993499a293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.080324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:35.099197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-34337cb5-4e3b-48b7-b372-47993499a293 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:35.100681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-34337cb5-4e3b-48b7-b372-47993499a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:35.251960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1090/2198] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:35.262825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b48ff5-aadf-4dd7-8ceb-4187e1a963cf None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.263162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b48ff5-aadf-4dd7-8ceb-4187e1a963cf None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.263479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b48ff5-aadf-4dd7-8ceb-4187e1a963cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.285951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-65b48ff5-aadf-4dd7-8ceb-4187e1a963cf None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:35.287184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-65b48ff5-aadf-4dd7-8ceb-4187e1a963cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:35.307288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1109/2199] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:35.311464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34337cb5-4e3b-48b7-b372-47993499a293 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:35.323837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34337cb5-4e3b-48b7-b372-47993499a293 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:35.331023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:35.331703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.331984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.332258 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.333017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:21:35.333326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:35.334921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.351779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1091/2200] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:21:35.354657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b48ff5-aadf-4dd7-8ceb-4187e1a963cf None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:35.366511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b48ff5-aadf-4dd7-8ceb-4187e1a963cf None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:35.373945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:35.374628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.374949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.375185 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.375887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:35.377840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.412622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1110/2201] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:35 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:35.416351 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:35.427611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446230fc-8e08-45d7-9134-cfb407f4e865 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:35.498898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:35.499592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.499920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.500112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.500870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:35.502766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.529492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1092/2202] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:35 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:21:35.533483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:35.545725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a80fd2-cd00-48a6-b726-4c469141c36d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:35.551028 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:35.551703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.551987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.552258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.553021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:35.554913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.566464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1111/2203] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:21:35.570232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:35.580346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d076fc-67e2-4668-ad0c-fc2134bd2776 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:35.585066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a28d93-a256-460b-839a-fa62a1d33c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:35.585733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a28d93-a256-460b-839a-fa62a1d33c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.586018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a28d93-a256-460b-839a-fa62a1d33c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.586295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a28d93-a256-460b-839a-fa62a1d33c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.587030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a28d93-a256-460b-839a-fa62a1d33c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:35.588664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a28d93-a256-460b-839a-fa62a1d33c4c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.603558 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1093/2204] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:35.605808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:35.618828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a798a-3de9-422f-8290-e1dd6a1baebf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:35.624435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:35.625722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.626108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.626534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] PATH_INFO: `/v3/projects/271787c093a4446dbf9ec777481862c3/users/8e4ed63bbae44810adfdf826e0b2ea9b/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.648444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] RBAC: Authorizing `identity:create_grant(project_id=271787c093a4446dbf9ec777481862c3, user_id=8e4ed63bbae44810adfdf826e0b2ea9b, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:35.654603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.699350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5faae-a8d6-447b-b00c-4a4604998f21 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.699779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5faae-a8d6-447b-b00c-4a4604998f21 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.700199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5faae-a8d6-447b-b00c-4a4604998f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.702914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1112/2205] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:35 2022] PUT /identity/v3/projects/271787c093a4446dbf9ec777481862c3/users/8e4ed63bbae44810adfdf826e0b2ea9b/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:35.734630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8aa5faae-a8d6-447b-b00c-4a4604998f21 None None] MFA Rules not processed for user `8e4ed63bbae44810adfdf826e0b2ea9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:35.735992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8aa5faae-a8d6-447b-b00c-4a4604998f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:35.803328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:35.812260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1094/2206] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:35.821453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:35.823278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:35.890734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a719e4-1203-45c2-93d6-3e093d02463d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:35.895616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d335bc-c6f7-417c-84ed-f32f3ebeb26e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:35.896429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d335bc-c6f7-417c-84ed-f32f3ebeb26e service glance] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:35.896842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d335bc-c6f7-417c-84ed-f32f3ebeb26e service glance] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:35.897231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d335bc-c6f7-417c-84ed-f32f3ebeb26e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:35.908542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d335bc-c6f7-417c-84ed-f32f3ebeb26e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:35.910771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d335bc-c6f7-417c-84ed-f32f3ebeb26e service glance] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:35.916327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:35.923758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1113/2207] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:21:36.295614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:36.296167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:36.296413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:36.322341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] MFA Rules not processed for user `3431c8ec634140039928dbeba141bc01`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:36.324023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:36.417101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1095/2208] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:37.885239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74f3d-8f12-4eb3-a899-4d5920e0341a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:37.885845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74f3d-8f12-4eb3-a899-4d5920e0341a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:37.885845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74f3d-8f12-4eb3-a899-4d5920e0341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:37.922140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-d0d74f3d-8f12-4eb3-a899-4d5920e0341a None None] MFA Rules not processed for user `dfc2fed192eb456d87b711d0d44e107f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:37.923618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d0d74f3d-8f12-4eb3-a899-4d5920e0341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:38.017428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1114/2209] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:38.021808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:38.023806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:38.127782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182861ec-1792-4258-a1b8-d30734a58589 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:38.141222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:38.142411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:38.142722 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:38.143074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:38.143812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:38.146718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:38.186824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1096/2210] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:21:38 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:38.213037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d74f3d-8f12-4eb3-a899-4d5920e0341a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:38.238347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d74f3d-8f12-4eb3-a899-4d5920e0341a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:38.327450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:38.328539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:38.329132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:38.329631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:38.330738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:38.334218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:38.366991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1115/2211] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:21:38 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:21:38.371926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:38.384498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8f98a-25fa-4b89-b18f-6526ba99c710 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:38.389913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:38.390439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:38.390845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:38.391225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:38.392070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:38.394851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:38.408283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1097/2212] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:38.412872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:38.427309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2e1b1-cead-4fe6-a691-cebd1e3a1413 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:38.449858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:38.457993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:38.458386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:38.462908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] PATH_INFO: `/v3/projects/74e7fdc61338416cacf6adea0e5f7c47/users/cf4cc5c20f714e38aa617db09cf938f0/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:38.484037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e7fdc61338416cacf6adea0e5f7c47, user_id=cf4cc5c20f714e38aa617db09cf938f0, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:38.495682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:38.518075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1116/2213] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:38 2022] PUT /identity/v3/projects/74e7fdc61338416cacf6adea0e5f7c47/users/cf4cc5c20f714e38aa617db09cf938f0/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:38.522919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:38.534148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5007ea-de37-4971-8cf9-1034b0375e68 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:38.613823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36ca9e-641e-4fce-805f-d5f6aef64626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:38.614789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36ca9e-641e-4fce-805f-d5f6aef64626 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:38.615328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36ca9e-641e-4fce-805f-d5f6aef64626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:38.615610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36ca9e-641e-4fce-805f-d5f6aef64626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:38.616340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36ca9e-641e-4fce-805f-d5f6aef64626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:38.619021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36ca9e-641e-4fce-805f-d5f6aef64626 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:38.633155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1098/2214] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:21:38.635388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:38.647353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b5718-4b12-4a8d-b7e4-a2cdd7162f75 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:38.652054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:38.653070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:38.653423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:38.653702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] PATH_INFO: `/v3/projects/74e7fdc61338416cacf6adea0e5f7c47/users/cf4cc5c20f714e38aa617db09cf938f0/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:38.655692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e7fdc61338416cacf6adea0e5f7c47, user_id=cf4cc5c20f714e38aa617db09cf938f0, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:38.658388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:38.670472 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1117/2215] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:21:38 2022] PUT /identity/v3/projects/74e7fdc61338416cacf6adea0e5f7c47/users/cf4cc5c20f714e38aa617db09cf938f0/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:21:45.469093 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92dd1-32d6-4ece-a81a-794551b3f233 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:45.470057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92dd1-32d6-4ece-a81a-794551b3f233 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:45.470605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92dd1-32d6-4ece-a81a-794551b3f233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:45.507237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-17f92dd1-32d6-4ece-a81a-794551b3f233 None None] MFA Rules not processed for user `cf4cc5c20f714e38aa617db09cf938f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:45.508577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-17f92dd1-32d6-4ece-a81a-794551b3f233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:45.512913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:45.599961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1099/2216] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:45.616904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:21:45.620507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:45.687522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:45.768776 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76774e-ffbd-4e93-849d-2bd498dbfa85 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:21:45.776311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:21:45.777088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:45.777358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:45.777634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:45.869187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:21:45.870842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:21:45.886952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1118/2217] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:21:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:21:46.821353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312b20-c35b-4912-b0ca-ac14de259445 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:21:46.822219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312b20-c35b-4912-b0ca-ac14de259445 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:21:46.822742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27312b20-c35b-4912-b0ca-ac14de259445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:21:46.849132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-27312b20-c35b-4912-b0ca-ac14de259445 None None] MFA Rules not processed for user `b97b26dd3bb5427a8edfe4eb653b945b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:21:46.850361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-27312b20-c35b-4912-b0ca-ac14de259445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:21:46.930168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1100/2218] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:21:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:21:55.877465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=50 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:21:56.922213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=52 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:03.781479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:03.782997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:03.876753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1baacce-1232-4750-ab4b-5f81699b32be service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:03.882162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:03.882817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:03.883102 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:03.883378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:03.885744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:03.898129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:03.899720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:03.911184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1119/2219] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:06.387396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90451533-3119-41f2-a13e-27746871c551 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:06.387892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90451533-3119-41f2-a13e-27746871c551 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:06.388030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90451533-3119-41f2-a13e-27746871c551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:06.446228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-90451533-3119-41f2-a13e-27746871c551 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:06.448050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-90451533-3119-41f2-a13e-27746871c551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:06.517420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1101/2220] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:22:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:06.563831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:06.565367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:06.660518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926c9bd-6402-48ca-a4d8-7515fff538e7 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:06.665942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:06.666704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:06.667163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:06.667449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:06.676522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:06.678166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:06.688015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1120/2221] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:22:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:22:07.274483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64b787-57a6-4839-a734-b493811579f2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:07.275274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64b787-57a6-4839-a734-b493811579f2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:07.275550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64b787-57a6-4839-a734-b493811579f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:07.293525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ae64b787-57a6-4839-a734-b493811579f2 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:07.294743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ae64b787-57a6-4839-a734-b493811579f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:07.301887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1102/2222] 10.222.0.52 () {58 vars in 967 bytes} [Fri Jun 24 09:22:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:07.316302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:07.326845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94db70-d31b-4ab7-bcf6-6150167af27a service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:07.332229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:07.332905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:07.333188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:07.333463 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:07.342544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:07.344227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:07.353467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1121/2223] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:22:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:22:07.554869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e73168-9bf5-437e-be0a-04027abd5859 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:07.555356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e73168-9bf5-437e-be0a-04027abd5859 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:07.555442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e73168-9bf5-437e-be0a-04027abd5859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:07.574684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-45e73168-9bf5-437e-be0a-04027abd5859 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:07.576505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-45e73168-9bf5-437e-be0a-04027abd5859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:07.584997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1103/2224] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:22:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:07.597030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:07.608388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad72f96-fc4f-4c98-9727-6f32e38b1067 service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:07.613481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:07.614217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:07.614561 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:07.614921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:07.623854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:07.625544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:07.633766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1122/2225] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:22:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:22:07.988983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:07.989278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:07.989557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:08.008470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:08.010382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:08.017690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1104/2226] 10.222.0.52 () {58 vars in 967 bytes} [Fri Jun 24 09:22:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:08.032519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:08.046876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcd788-c0f9-4485-898a-f35c077d9041 service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:08.051884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9170be-967e-4bd1-98fc-083e16b4a329 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:08.052531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9170be-967e-4bd1-98fc-083e16b4a329 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:08.052843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9170be-967e-4bd1-98fc-083e16b4a329 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:08.053117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9170be-967e-4bd1-98fc-083e16b4a329 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:08.062798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9170be-967e-4bd1-98fc-083e16b4a329 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:08.064349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9170be-967e-4bd1-98fc-083e16b4a329 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:08.069215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:08.075515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1123/2227] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:22:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:22:17.469681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:17.471125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:17.534813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ba54b-4802-4bfa-9779-2ea8600e1a1b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:17.539428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:17.540872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:17.541152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:17.541428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] PATH_INFO: `/v3/users/3431c8ec634140039928dbeba141bc01` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:17.542619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3431c8ec634140039928dbeba141bc01)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:17.544637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:17.589835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:17.591300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3431c8ec634140039928dbeba141bc01'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:17.598679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3431c8ec634140039928dbeba141bc01'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:17.607340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3431c8ec634140039928dbeba141bc01'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:17.608170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3431c8ec634140039928dbeba141bc01'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:17.608910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3431c8ec634140039928dbeba141bc01'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:17.629318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1105/2228] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:22:17 2022] DELETE /identity/v3/users/3431c8ec634140039928dbeba141bc01 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:22:17.633838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:17.634336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:17.634440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:17.651316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:17.652499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:17.710439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1124/2229] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:17.720814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:17.722209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:17.779307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df988-0c3c-42ef-8ace-a2e84864085f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:17.782969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dea422-1e98-4dc5-b95d-3a7dc1f221e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:17.783610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dea422-1e98-4dc5-b95d-3a7dc1f221e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:17.783895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dea422-1e98-4dc5-b95d-3a7dc1f221e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:17.784170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dea422-1e98-4dc5-b95d-3a7dc1f221e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:17.794524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dea422-1e98-4dc5-b95d-3a7dc1f221e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:17.796062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dea422-1e98-4dc5-b95d-3a7dc1f221e1 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:17.805726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1106/2230] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:17.978389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:17.979864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:17.991507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26178c9b-4225-4879-8c78-09c005acbbcb None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:17.996865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:17.998630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:17.998919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:17.999198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] PATH_INFO: `/v3/projects/1fcfd950d921449f98b14d6c63bf8793` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.000531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fcfd950d921449f98b14d6c63bf8793)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.003115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.052249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fcfd950d921449f98b14d6c63bf8793', 'request_id': 'req-30df5818-db60-4781-970c-a57da33af441'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:18.060281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fcfd950d921449f98b14d6c63bf8793', 'request_id': 'req-30df5818-db60-4781-970c-a57da33af441'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:18.065893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fcfd950d921449f98b14d6c63bf8793', 'request_id': 'req-30df5818-db60-4781-970c-a57da33af441'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:18.077032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] The token cache is being invalidate because project 1fcfd950d921449f98b14d6c63bf8793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:22:18.077318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:18.077941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-30df5818-db60-4781-970c-a57da33af441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:18.079478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1125/2231] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:22:17 2022] DELETE /identity/v3/projects/1fcfd950d921449f98b14d6c63bf8793 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:22:18.084356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b090d-3629-4060-810a-3658238ba08c None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.084671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b090d-3629-4060-810a-3658238ba08c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.084948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b090d-3629-4060-810a-3658238ba08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.102360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e07b090d-3629-4060-810a-3658238ba08c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:18.104077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e07b090d-3629-4060-810a-3658238ba08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:18.163613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1107/2232] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:18.172104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c7236-7f6e-4ac5-9dad-06737b5b3b0c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.172543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c7236-7f6e-4ac5-9dad-06737b5b3b0c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.172862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c7236-7f6e-4ac5-9dad-06737b5b3b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.192040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-456c7236-7f6e-4ac5-9dad-06737b5b3b0c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:18.193266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-456c7236-7f6e-4ac5-9dad-06737b5b3b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:18.201119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1126/2233] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:18.205218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b090d-3629-4060-810a-3658238ba08c None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:18.214799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b090d-3629-4060-810a-3658238ba08c None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:18.219432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:18.220078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.220357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.220664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.221376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:22:18.221680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.223257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.235583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1108/2234] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:22:18.239810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c7236-7f6e-4ac5-9dad-06737b5b3b0c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:18.249775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c7236-7f6e-4ac5-9dad-06737b5b3b0c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:18.254426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:18.255109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.255395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.255707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.256402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.258331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.290856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1127/2235] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:22:18 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:22:18.295059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:18.305727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6ec7-9314-4f79-beed-0a235b75623d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:18.354347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:18.365385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:18.366233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.366678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.367200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.367925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.370338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.394814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1109/2236] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:22:18 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:22:18.395956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:18.406936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584800e-0d03-4a4d-b5b5-787ea2b5f8de admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:18.411493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:18.412128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.412412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.412724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.413449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.415050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.425252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1128/2237] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:22:18.429304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:18.439898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce52789-12f4-46e6-9552-6b6a3f408e16 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:18.444443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7001d62-e128-4de7-b283-b375bf1152b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:18.445250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7001d62-e128-4de7-b283-b375bf1152b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.445543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7001d62-e128-4de7-b283-b375bf1152b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.445820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7001d62-e128-4de7-b283-b375bf1152b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.447231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7001d62-e128-4de7-b283-b375bf1152b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.448836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7001d62-e128-4de7-b283-b375bf1152b5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.458551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1110/2238] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:22:18.462469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:18.472150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738fdc1-9b32-4388-84d9-f791797c8702 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:18.476935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:18.477836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.478119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.478395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] PATH_INFO: `/v3/projects/267db8931ade4a2d8658ccf487dad772/users/8b60a3f963f34bbab49682c9d7a80bb5/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.492487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=267db8931ade4a2d8658ccf487dad772, user_id=8b60a3f963f34bbab49682c9d7a80bb5, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.495653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.508348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1129/2239] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:22:18 2022] PUT /identity/v3/projects/267db8931ade4a2d8658ccf487dad772/users/8b60a3f963f34bbab49682c9d7a80bb5/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:22:18.513986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b923c-e036-427d-a54d-728885435c08 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.514545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b923c-e036-427d-a54d-728885435c08 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.514545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b923c-e036-427d-a54d-728885435c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.531924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3d5b923c-e036-427d-a54d-728885435c08 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:18.533158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3d5b923c-e036-427d-a54d-728885435c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:18.593857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1111/2240] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:18.603998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:18.605639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:18.627164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:18.663503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aef120-45a5-4f75-b136-c488b469b56c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:18.667765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:18.668404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:18.668715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:18.668991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:18.677807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:18.679481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:18.688716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1130/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:19.282925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512e5b2-1513-4000-a1b2-d44c934c8148 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:19.283213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512e5b2-1513-4000-a1b2-d44c934c8148 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:19.283492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512e5b2-1513-4000-a1b2-d44c934c8148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:19.309504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-8512e5b2-1513-4000-a1b2-d44c934c8148 None None] MFA Rules not processed for user `8b60a3f963f34bbab49682c9d7a80bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:19.310713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-8512e5b2-1513-4000-a1b2-d44c934c8148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:19.385010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1112/2242] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:19.395740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:19.397514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:19.484025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d39e92-a643-48e5-8dc9-17dda88c0b19 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:19.487606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49125-4d6b-4d50-b32d-af1e29e431b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:19.488245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49125-4d6b-4d50-b32d-af1e29e431b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:19.488525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49125-4d6b-4d50-b32d-af1e29e431b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:19.488825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49125-4d6b-4d50-b32d-af1e29e431b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:19.497085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49125-4d6b-4d50-b32d-af1e29e431b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:19.498514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49125-4d6b-4d50-b32d-af1e29e431b2 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:19.507478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1131/2243] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:21.895121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:21.895633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:21.895766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:21.916010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] MFA Rules not processed for user `8b60a3f963f34bbab49682c9d7a80bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:21.917163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:21.924063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1113/2244] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:22.654093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac191a-dc89-4d8e-9861-40550c10c934 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:22.654595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac191a-dc89-4d8e-9861-40550c10c934 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:22.654705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac191a-dc89-4d8e-9861-40550c10c934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:22.672818 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-77ac191a-dc89-4d8e-9861-40550c10c934 None None] MFA Rules not processed for user `dfc2fed192eb456d87b711d0d44e107f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:22.674197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-77ac191a-dc89-4d8e-9861-40550c10c934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:22.734816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1132/2245] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:31.910432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:32.727905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=77 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:34.026943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:34.028941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:34.048039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3220c-2917-4852-a9e3-e86aba2556d7 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:34.052659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:34.053539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.054079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.054353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] PATH_INFO: `/v3/users/8e4ed63bbae44810adfdf826e0b2ea9b` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.056177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e4ed63bbae44810adfdf826e0b2ea9b)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:34.058129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:34.110117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e4ed63bbae44810adfdf826e0b2ea9b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.118277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e4ed63bbae44810adfdf826e0b2ea9b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.125269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e4ed63bbae44810adfdf826e0b2ea9b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.125900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e4ed63bbae44810adfdf826e0b2ea9b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.126508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e4ed63bbae44810adfdf826e0b2ea9b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.143242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1114/2246] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:22:34 2022] DELETE /identity/v3/users/8e4ed63bbae44810adfdf826e0b2ea9b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:22:34.147642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.148139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.148251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.166049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:34.167223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:34.225574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1133/2247] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:34.235888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:34.237306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:34.301096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176bc58-4d0d-4e5f-b51a-d90b77ba9dc5 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:34.305605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2de6-2185-4909-8fa8-afd0d7669363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:34.306252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2de6-2185-4909-8fa8-afd0d7669363 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.306534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2de6-2185-4909-8fa8-afd0d7669363 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.306810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2de6-2185-4909-8fa8-afd0d7669363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.315943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f2de6-2185-4909-8fa8-afd0d7669363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:34.317388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f2de6-2185-4909-8fa8-afd0d7669363 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:34.327396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1115/2248] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:34.616616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:34.618394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:34.635335 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb004a-be04-42ac-9431-51ac2ad1c1f6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:34.641117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:34.641995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.642279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.642618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] PATH_INFO: `/v3/projects/271787c093a4446dbf9ec777481862c3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.643790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=271787c093a4446dbf9ec777481862c3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:34.645993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:34.707912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271787c093a4446dbf9ec777481862c3', 'request_id': 'req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.718127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271787c093a4446dbf9ec777481862c3', 'request_id': 'req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.724446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271787c093a4446dbf9ec777481862c3', 'request_id': 'req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.735449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] The token cache is being invalidate because project 271787c093a4446dbf9ec777481862c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:22:34.735734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.736549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-c1ab738b-df57-49fc-b1fa-e443f9d0ae01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:22:34.738112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1134/2249] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:22:34 2022] DELETE /identity/v3/projects/271787c093a4446dbf9ec777481862c3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:22:34.743209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea122d60-cdb9-42ca-8680-2661251562bf None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.743500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea122d60-cdb9-42ca-8680-2661251562bf None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.743781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea122d60-cdb9-42ca-8680-2661251562bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.762951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ea122d60-cdb9-42ca-8680-2661251562bf None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:34.764169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ea122d60-cdb9-42ca-8680-2661251562bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:34.823399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1116/2250] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:34.832128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca1078-2fe2-42e0-a122-69b60e8040ea None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.832416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca1078-2fe2-42e0-a122-69b60e8040ea None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.832874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca1078-2fe2-42e0-a122-69b60e8040ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.851133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f0ca1078-2fe2-42e0-a122-69b60e8040ea None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:34.852380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f0ca1078-2fe2-42e0-a122-69b60e8040ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:34.860505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1135/2251] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:34.864756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea122d60-cdb9-42ca-8680-2661251562bf None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:34.874300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea122d60-cdb9-42ca-8680-2661251562bf None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:34.878829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:34.879475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.879759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.880034 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.880789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:22:34.881101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:34.882716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:34.895585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1117/2252] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:22:34.900288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca1078-2fe2-42e0-a122-69b60e8040ea None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:34.909596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca1078-2fe2-42e0-a122-69b60e8040ea None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:34.914169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:34.914825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:34.915096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:34.915370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:34.916062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:34.917983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:34.944817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1136/2253] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:22:34 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:22:34.949388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:34.954478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:34.960777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5acea5-faf6-4f98-abbe-63ee0b58aff7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:35.017920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:35.023382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:35.023382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:35.023382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:35.023382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:35.023382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:35.043100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1118/2254] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:22:34 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:22:35.047143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:35.058216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2867766-1b5f-48b1-9e91-2254132d2d2b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:35.062923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:35.063574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:35.063856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:35.064131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:35.064895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:35.066486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:35.077517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1137/2255] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:22:35.081278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:35.091470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdae998-98ab-4243-896c-6c6cb5db1a55 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:35.096318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b261ce-aa21-4e7f-a7f1-cfed397a9f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:35.097039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b261ce-aa21-4e7f-a7f1-cfed397a9f4c admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:35.097322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b261ce-aa21-4e7f-a7f1-cfed397a9f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:35.097628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b261ce-aa21-4e7f-a7f1-cfed397a9f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:35.098383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b261ce-aa21-4e7f-a7f1-cfed397a9f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:35.099966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b261ce-aa21-4e7f-a7f1-cfed397a9f4c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:35.109854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1119/2256] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:22:35.113695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:35.124063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d493a9-2cb8-4c19-930f-a5f3841dd2ae admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:35.128615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:35.129494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:35.130058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:35.130058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] PATH_INFO: `/v3/projects/8827e0b03a834c1eaa62a846c7b19b1e/users/467ecd55b48d4caf9f1a71f4d49140be/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:35.145716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8827e0b03a834c1eaa62a846c7b19b1e, user_id=467ecd55b48d4caf9f1a71f4d49140be, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:35.148474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:35.161503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1138/2257] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:22:35 2022] PUT /identity/v3/projects/8827e0b03a834c1eaa62a846c7b19b1e/users/467ecd55b48d4caf9f1a71f4d49140be/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:22:35.166414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7d29-9f6d-4076-853c-1e63424d671c None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:35.166706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7d29-9f6d-4076-853c-1e63424d671c None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:35.166985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7d29-9f6d-4076-853c-1e63424d671c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:35.186172 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-588d7d29-9f6d-4076-853c-1e63424d671c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:35.186751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-588d7d29-9f6d-4076-853c-1e63424d671c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:35.247241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1120/2258] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:35.256729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:35.258819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:35.285223 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:35.332699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc25e48-0d19-4e26-8bcd-66b94d962537 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:35.336416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:35.337587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:35.337873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:35.338151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:35.347402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:35.348861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:35.358654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1139/2259] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:39.646188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8a3e0-c06b-469f-9a49-95ed9540d16d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:39.646711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8a3e0-c06b-469f-9a49-95ed9540d16d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:39.646916 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8a3e0-c06b-469f-9a49-95ed9540d16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:39.681788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-4df8a3e0-c06b-469f-9a49-95ed9540d16d None None] MFA Rules not processed for user `467ecd55b48d4caf9f1a71f4d49140be`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:39.682998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4df8a3e0-c06b-469f-9a49-95ed9540d16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:39.773749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1121/2260] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:39.778876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:39.792781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d0445-e5cb-4d55-90de-80f8eaafbb4f service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:39.798211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:39.798986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:39.799279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:39.799555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:39.800279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:39.802277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:39.835488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1140/2261] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:22:39 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:22:39.841511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8a3e0-c06b-469f-9a49-95ed9540d16d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:39.856594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8a3e0-c06b-469f-9a49-95ed9540d16d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:39.867609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:39.935656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:39.936401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:39.936738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:39.937018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:39.937762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:39.939711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:39.971367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1122/2262] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:22:39 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:22:39.975799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:39.986836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5d00-097d-43a6-a5a3-033331cdb5ca admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:39.991761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:39.993027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:39.993475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:39.993910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:39.994803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:39.996871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:40.010964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1141/2263] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:22:40.015006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:40.028012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b0397-a361-42d9-b40e-16bcb4b8d557 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:40.037918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:40.040774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:40.041173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:40.041460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] PATH_INFO: `/v3/projects/07aaa90ba73843959d4b19919b02cbf3/users/34839ad16eeb4a75ba121a5d32fc0b44/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:40.062562 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07aaa90ba73843959d4b19919b02cbf3, user_id=34839ad16eeb4a75ba121a5d32fc0b44, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:40.065683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:40.082800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1123/2264] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:22:40 2022] PUT /identity/v3/projects/07aaa90ba73843959d4b19919b02cbf3/users/34839ad16eeb4a75ba121a5d32fc0b44/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:22:40.087356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:40.098460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f652-27f7-4f96-8fab-8f01a1fe5834 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:40.210881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a4ebc-3d39-4bb6-ab8e-03aa89b31b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:40.213509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4ebc-3d39-4bb6-ab8e-03aa89b31b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:40.214356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4ebc-3d39-4bb6-ab8e-03aa89b31b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:40.214648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4ebc-3d39-4bb6-ab8e-03aa89b31b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:40.215409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4ebc-3d39-4bb6-ab8e-03aa89b31b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:40.217146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4ebc-3d39-4bb6-ab8e-03aa89b31b78 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:40.229828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1142/2265] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:22:40.234240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:40.245708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b63d75-8baa-4bfc-9c4e-726f16cf9ab1 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:40.250974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:40.251873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:40.252241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:40.252532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] PATH_INFO: `/v3/projects/07aaa90ba73843959d4b19919b02cbf3/users/34839ad16eeb4a75ba121a5d32fc0b44/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:40.254705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07aaa90ba73843959d4b19919b02cbf3, user_id=34839ad16eeb4a75ba121a5d32fc0b44, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:40.257698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:40.271704 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1124/2266] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:22:40 2022] PUT /identity/v3/projects/07aaa90ba73843959d4b19919b02cbf3/users/34839ad16eeb4a75ba121a5d32fc0b44/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:22:45.269933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cec5e0-e7a3-4023-83fd-ac06cecfd6b5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:45.270433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cec5e0-e7a3-4023-83fd-ac06cecfd6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:45.270521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cec5e0-e7a3-4023-83fd-ac06cecfd6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:45.304520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-18cec5e0-e7a3-4023-83fd-ac06cecfd6b5 None None] MFA Rules not processed for user `34839ad16eeb4a75ba121a5d32fc0b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:45.306891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-18cec5e0-e7a3-4023-83fd-ac06cecfd6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:45.396756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1143/2267] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:45.404833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:45.407175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:45.503873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091905d-251a-48ae-9ac5-9dcc0b8e8539 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:45.508666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:45.508809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:45.509082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:45.509355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:45.576660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:45.578851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:45.582738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:45.594788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1125/2268] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:45.694621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:45.694908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:45.695182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:45.719121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:45.719842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:45.725998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:22:45.794865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1144/2269] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:22:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:45.807410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:45.808898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:45.889851 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70510f7d-8623-4163-a0aa-90aa3080d8a7 service glance] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:45.897791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81e00-0cd9-4f24-ab17-cda2829722dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:45.898549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81e00-0cd9-4f24-ab17-cda2829722dd service swift] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:45.899365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81e00-0cd9-4f24-ab17-cda2829722dd service swift] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:45.899642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81e00-0cd9-4f24-ab17-cda2829722dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:45.908593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81e00-0cd9-4f24-ab17-cda2829722dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:45.910199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81e00-0cd9-4f24-ab17-cda2829722dd service swift] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:45.919563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1126/2270] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:22:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:22:51.748870 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:51.750395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:51.821413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e19439-9e0f-441b-8a2d-f9e104d8587a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:51.825831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c18e35-585a-4b1f-b6db-ea275cf654ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:51.826677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c18e35-585a-4b1f-b6db-ea275cf654ca service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:51.827139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c18e35-585a-4b1f-b6db-ea275cf654ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:51.828150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c18e35-585a-4b1f-b6db-ea275cf654ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:51.830956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-93c18e35-585a-4b1f-b6db-ea275cf654ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:51.911272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c18e35-585a-4b1f-b6db-ea275cf654ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:51.913036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c18e35-585a-4b1f-b6db-ea275cf654ca service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:51.952308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1145/2271] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:22:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:22:55.739470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d78897-6131-4ccb-887c-60ad6a449367 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:55.740010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d78897-6131-4ccb-887c-60ad6a449367 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:55.740113 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d78897-6131-4ccb-887c-60ad6a449367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:55.758569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c9d78897-6131-4ccb-887c-60ad6a449367 None None] MFA Rules not processed for user `467ecd55b48d4caf9f1a71f4d49140be`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:55.759252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c9d78897-6131-4ccb-887c-60ad6a449367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:55.766178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1127/2272] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:22:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:57.585967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:57.586477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:57.586560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:57.604385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:22:57.605698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:22:57.612864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1146/2273] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:22:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:22:57.625527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d78897-6131-4ccb-887c-60ad6a449367 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:22:57.637995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d78897-6131-4ccb-887c-60ad6a449367 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:22:57.643114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:22:57.643755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:22:57.644035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:22:57.644310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:22:57.653422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:22:57.655003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:22:57.663710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1128/2274] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:22:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:23:07.599694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=73 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:23:07.660891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=42 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:24:01.073489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:01.075177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:01.130297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50246c28-facb-4b77-a402-eeb2111ef31b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:01.134452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:01.135083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:01.135360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:01.135631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:01.137527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:01.196037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:01.197501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:01.206599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1147/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:07.335388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:07.345544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47285d-206f-413e-b24b-d7e5405ae7cb service swift] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:07.349218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:07.349850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:07.350134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:07.350408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:07.352196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:07.412794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:07.414234 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:07.423864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1129/2276] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:08.554233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:08.564078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700ec17-7528-4615-b975-60e43fcc3908 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:08.567724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:08.568372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:08.568692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:08.568978 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:08.571008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:08.641700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:08.643119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:08.654259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1148/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:12.156351 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:12.165482 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daef30-5df9-4789-bf60-6a47b49aaad3 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:12.170001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c44d32-1c71-4e9c-867e-c7a8026fa8ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:12.170634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c44d32-1c71-4e9c-867e-c7a8026fa8ad service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:12.170917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c44d32-1c71-4e9c-867e-c7a8026fa8ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:12.171193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c44d32-1c71-4e9c-867e-c7a8026fa8ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:12.172996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-55c44d32-1c71-4e9c-867e-c7a8026fa8ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:12.186998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c44d32-1c71-4e9c-867e-c7a8026fa8ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:12.188407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c44d32-1c71-4e9c-867e-c7a8026fa8ad service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:12.199655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1130/2278] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:14.505056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:14.506516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:14.520334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510e1df-9b5c-46d5-a346-7ec7dd2461f3 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:14.526354 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:14.526980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:14.527261 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:14.527535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:14.529296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:14.567808 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:24:14.587987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:14.589536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:14.598450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1149/2279] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:21.681052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:21.681539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:21.681643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:21.698836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] MFA Rules not processed for user `f8333f6a9dab4618bc0e6b2060f15250`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:24:21.700082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:21.706927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1131/2280] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:24:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:24:22.014503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:22.024208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea9d18-6684-4d9c-bf43-5e367aa417f7 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:22.028458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:22.029205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:22.029496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:22.029835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:22.031657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:22.041693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:22.043111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:22.051038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1150/2281] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:22.365920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:22.367407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:22.438809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968f890-32f1-42d6-b00c-efafc4d958be None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:22.443641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4327c2-6e29-43dd-82a7-eb2e94c9a504 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:22.444505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4327c2-6e29-43dd-82a7-eb2e94c9a504 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:22.444821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4327c2-6e29-43dd-82a7-eb2e94c9a504 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:22.445097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4327c2-6e29-43dd-82a7-eb2e94c9a504 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:22.447483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4f4327c2-6e29-43dd-82a7-eb2e94c9a504 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:22.458366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4327c2-6e29-43dd-82a7-eb2e94c9a504 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:22.459921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4327c2-6e29-43dd-82a7-eb2e94c9a504 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:22.469586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1132/2282] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:29.801034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:29.811979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22eecf3-55c4-4c3a-9abd-1a3bfcd1762a service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:29.815692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:29.816333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:29.816648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:29.816932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:29.818851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:29.829188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:29.830702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:29.839538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1151/2283] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:32.091335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f7e33-44c1-43f3-8e38-d78867014471 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:32.091837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f7e33-44c1-43f3-8e38-d78867014471 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:32.091943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f7e33-44c1-43f3-8e38-d78867014471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:32.098044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=70 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:24:32.112668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-519f7e33-44c1-43f3-8e38-d78867014471 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:24:32.113866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-519f7e33-44c1-43f3-8e38-d78867014471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:32.121207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1133/2284] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:24:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:24:32.162334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:32.175214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b6b5f-f7af-48ac-9f6e-6e967b971251 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:32.180473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:32.181178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:32.181733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:32.182299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:32.190723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:32.192307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:32.201158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1152/2285] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:24:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:24:36.368449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f7e33-44c1-43f3-8e38-d78867014471 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:36.388514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f7e33-44c1-43f3-8e38-d78867014471 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:36.395650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349dffa-a08c-4535-85b0-3e0c23b298ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:36.396315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349dffa-a08c-4535-85b0-3e0c23b298ba service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:36.396753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349dffa-a08c-4535-85b0-3e0c23b298ba service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:36.396902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349dffa-a08c-4535-85b0-3e0c23b298ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:36.400864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3349dffa-a08c-4535-85b0-3e0c23b298ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:36.496848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349dffa-a08c-4535-85b0-3e0c23b298ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:36.499127 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349dffa-a08c-4535-85b0-3e0c23b298ba service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:36.522740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1134/2286] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:39.189394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:39.203356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37d9d-aae1-4498-976f-d9ba64db7308 service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:39.221232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464c717-1e6c-4124-a5ae-3b7a07a409c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:39.222672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c717-1e6c-4124-a5ae-3b7a07a409c9 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:39.224214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c717-1e6c-4124-a5ae-3b7a07a409c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:39.225313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c717-1e6c-4124-a5ae-3b7a07a409c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:39.227945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b464c717-1e6c-4124-a5ae-3b7a07a409c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:39.251292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464c717-1e6c-4124-a5ae-3b7a07a409c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:39.253090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464c717-1e6c-4124-a5ae-3b7a07a409c9 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:39.266745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1153/2287] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:24:46.507804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=30 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:24:49.261020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=34 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:24:49.817663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5e31-cd88-4ca9-848f-414fdff01f12 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:49.818134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5e31-cd88-4ca9-848f-414fdff01f12 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:49.818325 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5e31-cd88-4ca9-848f-414fdff01f12 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:49.826979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcccb-a9b9-4c2f-a767-e19d9ec323d9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:49.827578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcccb-a9b9-4c2f-a767-e19d9ec323d9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:49.828060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcccb-a9b9-4c2f-a767-e19d9ec323d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:49.828269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1135/2288] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:24:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:24:49.868461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-16dfcccb-a9b9-4c2f-a767-e19d9ec323d9 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:24:49.869829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-16dfcccb-a9b9-4c2f-a767-e19d9ec323d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:49.879525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1154/2289] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:24:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:24:49.895340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5e31-cd88-4ca9-848f-414fdff01f12 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:49.897353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-108d5e31-cd88-4ca9-848f-414fdff01f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:49.916424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5e31-cd88-4ca9-848f-414fdff01f12 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:49.920496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae1c4b-f887-4569-8e1c-334e83f9044b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:49.921362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1c4b-f887-4569-8e1c-334e83f9044b service glance] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:49.921808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1c4b-f887-4569-8e1c-334e83f9044b service glance] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:49.922086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1c4b-f887-4569-8e1c-334e83f9044b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:49.933192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae1c4b-f887-4569-8e1c-334e83f9044b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:49.934632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae1c4b-f887-4569-8e1c-334e83f9044b service glance] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:49.945867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1136/2290] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:24:50.010824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb18e4-4192-49d4-adea-3103eddc0b1b None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:50.011118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb18e4-4192-49d4-adea-3103eddc0b1b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:50.011401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb18e4-4192-49d4-adea-3103eddc0b1b None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:50.012812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1155/2291] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:24:50.017682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcafbc-be8c-4b29-99c8-40a07be48b7d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:50.017999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcafbc-be8c-4b29-99c8-40a07be48b7d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:50.018281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcafbc-be8c-4b29-99c8-40a07be48b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:50.038646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-81bcafbc-be8c-4b29-99c8-40a07be48b7d None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:24:50.039802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-81bcafbc-be8c-4b29-99c8-40a07be48b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:50.050670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1137/2292] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:24:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:24:50.062989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb18e4-4192-49d4-adea-3103eddc0b1b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:50.074530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb18e4-4192-49d4-adea-3103eddc0b1b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:50.078070 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a06a4-8c1d-47af-be68-acb07085e1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:50.078694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a06a4-8c1d-47af-be68-acb07085e1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:50.078977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a06a4-8c1d-47af-be68-acb07085e1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:50.079251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a06a4-8c1d-47af-be68-acb07085e1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:50.086159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a06a4-8c1d-47af-be68-acb07085e1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:50.087565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a06a4-8c1d-47af-be68-acb07085e1d4 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:50.095915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1156/2293] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:24:50.175761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f86b04-8d39-4592-9517-0b9756554cf9 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:50.176064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f86b04-8d39-4592-9517-0b9756554cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:50.176339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f86b04-8d39-4592-9517-0b9756554cf9 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:50.177737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1138/2294] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 09:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:24:50.183628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:50.183920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:50.184197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:50.202635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:24:50.203809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:24:50.212938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1157/2295] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 09:24:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:24:50.223415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f86b04-8d39-4592-9517-0b9756554cf9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:24:50.234365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f86b04-8d39-4592-9517-0b9756554cf9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:24:50.237985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:24:50.238611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:24:50.238922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:24:50.239197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:24:50.249803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:24:50.251209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:24:50.262118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1139/2296] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:24:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:25:00.198771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=18 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:25:00.257068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=15 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:26:45.961464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:26:45.962983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:26:45.976536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6497d97-9d3b-4569-86da-b4f2b30602ed None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:26:45.982434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:26:45.983137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:26:45.983422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:26:45.983712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:26:45.985455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:26:46.056546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:26:46.058248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:26:46.080709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1158/2297] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:26:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:26:56.066680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=28 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:40.965118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:40.966545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:27:40.981820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c0adf-7577-4e7e-a4a0-ffe7480d2e2a service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:40.987464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:40.988209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:40.988584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:40.988953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:40.990942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:27:41.003064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:41.005320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:41.028183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1140/2298] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:27:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:27:51.011867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:10.564156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:10.573346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bf4cc-056d-4431-b0ce-fc57733031cf service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:10.576957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:10.577592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:10.577889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:10.578155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:10.579993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:10.640360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:10.641814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:10.650606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1159/2299] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:28:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:28:10.716832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:10.725808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd16bf-768f-4386-86ff-c86a41087b45 service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:10.729459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:10.730084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:10.730476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:10.730839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:10.738816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:10.740217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:10.748749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1141/2300] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:28:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:28:20.646573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=28 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:20.744952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.077518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.090422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61da457-8f15-4c80-92bd-d26587e94145 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.095731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.096411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.096738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.097012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.104700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.106258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.133343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1160/2301] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:29:21.130726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=13 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:04.850625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:04.859841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893e3d6-9982-4099-a562-d05dfd07fc26 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:04.863381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:04.864012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:04.864292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:04.864639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:04.871676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:04.873240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:04.881026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1142/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:11.041650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:11.048571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbc6c6-a9c6-41df-a13a-f7d05aeaaca4 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:11.052156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:11.052820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:11.053100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:11.053629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:11.061874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:11.063277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:11.089852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1161/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:11.160476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:11.171272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f6061-cd20-4590-b9a3-fcdabd222791 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:11.175132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:11.176327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:11.176643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:11.176930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:11.189864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:11.192060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:11.203675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1143/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:11.252597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:11.263387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12598511-6754-47c5-b971-35f2058a7a3f service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:11.269519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:11.270212 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:11.271073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:11.271073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:11.278648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:11.281218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:11.283014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:11.291006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752055-cd2c-494b-bf35-6e8c6da0f2df service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:11.297538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:11.298204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:11.298485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:11.298759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:11.308942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:11.310823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:11.314200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1162/2305] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:11.340107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1144/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:30:11.452159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:11.462451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f12f-73d5-4153-9b4b-a98b4ff95ed1 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:11.466388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ef931-7241-4e80-8b9b-9b185f28f895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:11.467039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ef931-7241-4e80-8b9b-9b185f28f895 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:11.467346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ef931-7241-4e80-8b9b-9b185f28f895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:11.467624 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ef931-7241-4e80-8b9b-9b185f28f895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:11.478934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ef931-7241-4e80-8b9b-9b185f28f895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:11.480353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ef931-7241-4e80-8b9b-9b185f28f895 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:11.492312 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1163/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:21.337433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=25 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:21.491017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=32 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:22.236304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:22.247649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db22b2-87cf-4f8f-913c-d1955807111d service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:22.253941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:22.254604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:22.255090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:22.255855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:22.265834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:22.267444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:22.282946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1145/2308] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:30:26.666720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c350-9e7f-4674-8533-d21635e1e444 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:26.667216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c350-9e7f-4674-8533-d21635e1e444 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:26.667338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931c350-9e7f-4674-8533-d21635e1e444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:26.707988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0931c350-9e7f-4674-8533-d21635e1e444 None None] MFA Rules not processed for user `6ea7013dd9424b9db1798e81521ee903`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:26.709214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0931c350-9e7f-4674-8533-d21635e1e444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:26.716249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1164/2309] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:30:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:30:27.025492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:27.035260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87e555-b8b0-485b-9ae3-c90e79ab5621 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:27.038862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:27.039501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:27.039801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:27.040058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:27.041852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:27.051999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:27.053701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:27.061874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1146/2310] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:27.387571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931c350-9e7f-4674-8533-d21635e1e444 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:27.400291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931c350-9e7f-4674-8533-d21635e1e444 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:27.405892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:27.406540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:27.406824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:27.407100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:27.416609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:27.418273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:27.431829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1165/2311] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:35.027608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:35.048269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605cc86-2c00-47af-ba0c-e7777e1abd5b service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:35.054993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:35.055909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:35.056398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:35.056733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:35.058621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:35.071663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:35.073518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:35.085835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1147/2312] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:30:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:30:37.424527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=17 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:45.081193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=18 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:44.957138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:44.998866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870f850-acde-446d-b791-85ccead5bfc3 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:45.004307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:45.004987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:45.005271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:45.005546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:45.013217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:45.015348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:45.043116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1166/2313] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:31:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:31:55.040277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=27 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:04.476395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:33:04.534171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660c79b-04a8-4bfa-8580-3da4bf3417ef service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:33:04.539330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:33:04.539966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:04.540250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:04.540534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:04.548902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:33:04.550472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:33:04.560374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1148/2314] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:33:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:33:14.556286 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=27 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:12.413679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:12.415288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:12.454611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:12.504665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647a802-9be5-4116-a781-3e1fb593ed9e service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:12.508315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:12.509167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:12.509451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:12.509723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:12.511604 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:12.543713 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bcc25-9cc7-4662-a2d1-63837c67329b service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:12.549663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:12.550309 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:12.550591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:12.550867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:12.553047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:12.591651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:12.593114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:12.612340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1167/2315] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:12.625647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:12.627246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:12.637186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1149/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:12.657800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:12.666997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548d6a8-54c6-4a4f-b32b-a456edc7a5a1 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:12.670613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:12.671249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:12.671529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:12.671804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:12.680894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:12.682602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:12.710805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1168/2317] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:22.632444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=68 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:22.708225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=87 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:09.223092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:09.224708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:09.300257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b132b-db25-4d6c-8609-28be6f67bb08 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:09.303998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:09.304657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:09.304942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:09.305222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:09.307078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:09.316158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:09.317603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:09.325410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1150/2318] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:11.562456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:11.564036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:11.575779 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457c5d9-5ac9-4d6a-b2ca-545b60b547a5 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:11.579341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:11.579983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:11.580264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:11.580539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:11.582359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:11.612641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:11.614092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:11.627354 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1169/2319] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:19.321898 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=35 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:21.620569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=11 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:58.892435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:58.901586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04659752-344a-4c56-80b0-3ff608ed1d3b service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:58.905314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:58.905990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:58.906271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:58.906547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:58.908409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:58.922139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:58.923551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:58.934553 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1151/2320] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:08.929791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:12.967839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:12.978066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3e680-3fdf-4f40-a0fa-81861e3dfd0c service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:12.981628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:12.982266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:12.982544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:12.982815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:12.984733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:13.058949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:13.060432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:13.070585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1170/2321] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:13.114838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:13.124132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5708c-3bab-48db-a626-9a6b7ad9f24c service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:13.128438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9122b-eebb-400c-b30a-7ffac03af885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:13.129102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9122b-eebb-400c-b30a-7ffac03af885 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:13.129381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9122b-eebb-400c-b30a-7ffac03af885 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:13.129656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9122b-eebb-400c-b30a-7ffac03af885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:13.131538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9ee9122b-eebb-400c-b30a-7ffac03af885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:13.203835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9122b-eebb-400c-b30a-7ffac03af885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:13.205297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9122b-eebb-400c-b30a-7ffac03af885 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:13.214200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1152/2322] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:13.259135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:13.268002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4266ba-1d6b-4c04-a9d7-1298b1174627 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:13.271804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:13.272443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:13.272763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:13.273038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:13.274898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:13.338254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:13.339765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:13.349677 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1171/2323] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:13.599649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:13.599943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:13.600229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:13.661335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:13.662540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:13.726886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1153/2324] 10.222.0.52 () {58 vars in 967 bytes} [Fri Jun 24 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:13.771101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:13.772717 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:13.862631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c1094-7cc2-4484-83cf-a7a4ff531993 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:13.867547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b6bc9-903d-4ce8-97d8-4480c0c13d53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:13.868181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b6bc9-903d-4ce8-97d8-4480c0c13d53 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:13.868460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b6bc9-903d-4ce8-97d8-4480c0c13d53 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:13.868761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b6bc9-903d-4ce8-97d8-4480c0c13d53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:13.874692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:13.876938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b6bc9-903d-4ce8-97d8-4480c0c13d53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:13.878890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b6bc9-903d-4ce8-97d8-4480c0c13d53 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:13.887402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1172/2325] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:36:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:36:15.375414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:15.378289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:15.398193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407ecdc-211a-4ac2-a9a0-1d857c90da67 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:15.405746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:15.406633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:15.407471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:15.407838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:15.410610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:15.426310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:15.428282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:15.436072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e96a0-a6f2-4725-9043-ff128da3839f None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:15.436681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e96a0-a6f2-4725-9043-ff128da3839f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:15.436681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e96a0-a6f2-4725-9043-ff128da3839f None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:15.437990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1173/2326] 10.222.0.52 () {54 vars in 912 bytes} [Fri Jun 24 09:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:36:15.440402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1154/2327] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:15.444095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ded1ef-ed24-42dc-b65c-f066fec1d7f0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:15.444493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ded1ef-ed24-42dc-b65c-f066fec1d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:15.444896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ded1ef-ed24-42dc-b65c-f066fec1d7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:15.505960 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-51ded1ef-ed24-42dc-b65c-f066fec1d7f0 None None] MFA Rules not processed for user `505dda787dac43bca3595fa1c406a457`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:15.507179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-51ded1ef-ed24-42dc-b65c-f066fec1d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:15.570493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1174/2328] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:36:15.593730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:15.605984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91461239-410e-4a8b-bb8a-da595bafa168 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:15.611245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:15.612269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:15.612618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:15.612858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:15.624230 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:15.625852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:15.638307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1155/2329] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:17.823123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:17.823862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:17.824093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:17.855086 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] MFA Rules not processed for user `00556257371a41b39bce3ca8ee545238`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:17.856205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:17.865245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1175/2330] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:18.167651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:18.177784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43d980-595b-40db-9c08-82720199cb9e service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:18.181393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:18.182021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:18.182305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:18.182584 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:18.184437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:18.193830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:18.195268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:18.206021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1156/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:18.527589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:18.529284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:18.544186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915a9f4-c045-4b76-8090-d39c448b72d2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:18.550574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd6bdc-ef18-4b62-968c-6ed490522901 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:18.552299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd6bdc-ef18-4b62-968c-6ed490522901 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:18.552811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd6bdc-ef18-4b62-968c-6ed490522901 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:18.553093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd6bdc-ef18-4b62-968c-6ed490522901 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:18.555032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4dcd6bdc-ef18-4b62-968c-6ed490522901 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:18.568403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd6bdc-ef18-4b62-968c-6ed490522901 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:18.570021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd6bdc-ef18-4b62-968c-6ed490522901 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:18.583031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1176/2332] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:21.416017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:21.443259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971aec-19d1-4c3b-a5d4-7e0cd9fabcf6 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:21.458727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:21.462126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:21.462417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:21.463121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:21.469651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:21.495675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:21.501735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:21.509453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:21.529617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1157/2333] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:22.836310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37784ffc-db8f-4f59-8804-4207476dc967 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:22.836796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37784ffc-db8f-4f59-8804-4207476dc967 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:22.836796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37784ffc-db8f-4f59-8804-4207476dc967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:22.854413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-37784ffc-db8f-4f59-8804-4207476dc967 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:22.855627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-37784ffc-db8f-4f59-8804-4207476dc967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:22.863478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1177/2334] 10.222.0.52 () {58 vars in 967 bytes} [Fri Jun 24 09:36:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:22.875926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:22.887587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8b715-bd2e-4e9f-9ae3-b13ccd93aac0 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:22.892585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:22.893224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:22.893507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:22.893785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:22.901766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:22.903327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:22.911321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1158/2335] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:36:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:36:23.302741 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37784ffc-db8f-4f59-8804-4207476dc967 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:23.312438 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37784ffc-db8f-4f59-8804-4207476dc967 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:23.316047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:23.316723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:23.317007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:23.317304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:23.319791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:23.405103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:23.406572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:23.412370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:23.436329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1178/2336] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:27.060365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:27.082178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a70ab-0304-4a63-80f3-c3c3d1054dc2 service swift] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:27.098406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:27.099568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:27.100239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:27.101222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:27.104154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:27.130054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:27.132143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:27.145808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1159/2337] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:29.048137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:29.070273 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8340c5e-80f6-4859-b6e0-bcfa8b74e145 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:29.077269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:29.078494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:29.079004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:29.079483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:29.098057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:29.100514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:29.120902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1179/2338] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:30.641767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:30.654006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e090943-add7-4657-b974-5bf4d5fa777e service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:30.660740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:30.661391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:30.661675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:30.661962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:30.663755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:30.675730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:30.684946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:30.688076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1160/2339] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:30.755288 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:30.757575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:30.772244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95ed68-f03c-4d21-90c5-20a3b436ef1d service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:30.779182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:30.780267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:30.780634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:30.780975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] PATH_INFO: `/v3/users/467ecd55b48d4caf9f1a71f4d49140be` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:30.782235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=467ecd55b48d4caf9f1a71f4d49140be)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:30.784328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:30.840263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '467ecd55b48d4caf9f1a71f4d49140be'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:30.848015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '467ecd55b48d4caf9f1a71f4d49140be'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:30.855539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '467ecd55b48d4caf9f1a71f4d49140be'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:30.856218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '467ecd55b48d4caf9f1a71f4d49140be'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:30.856854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '467ecd55b48d4caf9f1a71f4d49140be'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:30.875160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1180/2340] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:36:30 2022] DELETE /identity/v3/users/467ecd55b48d4caf9f1a71f4d49140be => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:36:31.044360 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:31.056545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aa45d-0fd1-4984-a410-95280565416e service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:31.118162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:31.119012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:31.119296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:31.119572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] PATH_INFO: `/v3/projects/8827e0b03a834c1eaa62a846c7b19b1e` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:31.120778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8827e0b03a834c1eaa62a846c7b19b1e)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:31.122683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:31.172511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8827e0b03a834c1eaa62a846c7b19b1e', 'request_id': 'req-809c3795-9894-4db4-a3a1-a8dc14e13de2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.192059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8827e0b03a834c1eaa62a846c7b19b1e', 'request_id': 'req-809c3795-9894-4db4-a3a1-a8dc14e13de2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.198456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8827e0b03a834c1eaa62a846c7b19b1e', 'request_id': 'req-809c3795-9894-4db4-a3a1-a8dc14e13de2'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.210736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] The token cache is being invalidate because project 8827e0b03a834c1eaa62a846c7b19b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:36:31.211116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.211901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.213555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1161/2341] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:36:31 2022] DELETE /identity/v3/projects/8827e0b03a834c1eaa62a846c7b19b1e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:36:31.218109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:31.219620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:31.234769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98e59c-c8f2-4c29-b509-5b35a5ad1adc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:31.297308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:31.298176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:31.298459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:31.299267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] PATH_INFO: `/v3/users/34839ad16eeb4a75ba121a5d32fc0b44` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:31.313163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34839ad16eeb4a75ba121a5d32fc0b44)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:31.315661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:31.358195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:31.364827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34839ad16eeb4a75ba121a5d32fc0b44'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.373582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34839ad16eeb4a75ba121a5d32fc0b44'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.382346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34839ad16eeb4a75ba121a5d32fc0b44'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.383157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34839ad16eeb4a75ba121a5d32fc0b44'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.383769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34839ad16eeb4a75ba121a5d32fc0b44'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.400645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1181/2342] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:36:31 2022] DELETE /identity/v3/users/34839ad16eeb4a75ba121a5d32fc0b44 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:36:31.573380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:31.574981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:31.588095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c3795-9894-4db4-a3a1-a8dc14e13de2 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:31.635791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:31.649800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:31.651737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:31.651737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:31.651737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] PATH_INFO: `/v3/projects/07aaa90ba73843959d4b19919b02cbf3` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:31.662621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07aaa90ba73843959d4b19919b02cbf3)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:31.664628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:31.712424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07aaa90ba73843959d4b19919b02cbf3', 'request_id': 'req-a2223e5d-a160-4a0a-8bec-b736fa16812f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.719303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07aaa90ba73843959d4b19919b02cbf3', 'request_id': 'req-a2223e5d-a160-4a0a-8bec-b736fa16812f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.725233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07aaa90ba73843959d4b19919b02cbf3', 'request_id': 'req-a2223e5d-a160-4a0a-8bec-b736fa16812f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.736702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] The token cache is being invalidate because project 07aaa90ba73843959d4b19919b02cbf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:36:31.736985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.737637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:36:31.739163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1162/2343] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:36:31 2022] DELETE /identity/v3/projects/07aaa90ba73843959d4b19919b02cbf3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:36:41.397080 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:41.733755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:45.077069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:45.078481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:45.135094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50709ce0-1519-4e4a-95f2-9f9daea99827 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:45.182618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:45.183260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:45.183545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:45.183822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:45.185725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:45.254746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:45.256171 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:45.265015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1182/2344] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:55.260940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=73 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:38:10.505763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:10.507145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:10.577217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2223e5d-a160-4a0a-8bec-b736fa16812f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:10.582235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:10.582882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:10.583166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:10.583445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:10.585246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:10.597115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:10.598696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:10.609088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1163/2345] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:38:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:38:20.605058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=32 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:39:12.994339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:39:12.995841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:13.068950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:39:13.070949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:13.073333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172651-b561-4517-b257-4091300a5fa2 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:39:13.079161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:39:13.079576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:13.079856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:13.080127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:13.082146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:13.082859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db80691-fda9-4914-a988-3c5a5355f925 service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:39:13.086463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:39:13.087103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:13.087392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:13.087675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:13.089789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:13.156200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:39:13.157923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:39:13.161739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:39:13.163164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:39:13.167503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1183/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:39:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:39:13.183551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1164/2347] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:39:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:39:23.162752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=61 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:39:23.180306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=36 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:40:11.715769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:40:11.717277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:11.773327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37123353-f931-4c67-9677-33e5e4819321 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:40:11.776934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:40:11.777572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:11.777858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:11.778137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:11.779890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:11.793509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:40:11.794928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:40:11.805991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1184/2348] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:40:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:40:16.234333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:40:16.235739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:16.247841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6382a-57bf-44b9-8f3c-db3fc8eaa792 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:40:16.251396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:40:16.252028 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:16.252310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:16.252612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:16.254316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:16.263437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:40:16.264883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:40:16.272642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1165/2349] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:40:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:40:21.801320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=26 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:40:26.269260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:40:59.758872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:40:59.767545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6b6c1-6cee-4aec-a5eb-b4ad9e25651a service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:40:59.771278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:40:59.771907 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:59.772192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:59.772469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:59.774284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:59.787480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:40:59.788925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:40:59.800454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1185/2350] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:40:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:09.795393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:13.067946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:13.077745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca348b9d-5581-4314-a913-2da97794610c service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:13.081270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:13.081902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:13.082185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:13.082463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:13.084603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:13.155054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:13.156429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:13.174639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1166/2351] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:13.217335 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:13.226320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91622-548f-4231-98f4-3ef1f4aff998 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:13.229833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:13.230457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:13.230737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:13.231010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:13.232790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:13.307891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:13.309346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:13.327626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1186/2352] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:13.407834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:13.416439 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5aa4e0-14e9-4d66-90b7-9f97b90c728d service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:13.420012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:13.420668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:13.420949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:13.421221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:13.422958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:13.500943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:13.502356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:13.519894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1167/2353] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:23.324995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=36 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:23.517326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=77 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:45.691689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:45.700224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d8ab2-f045-4757-b6dc-6caa271d561d service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.703860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.704487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.704796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.705069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.713930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.715325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.741992 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1187/2354] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:55.739380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=13 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:14.640273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:14.650229 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb0267-79bf-4f54-a5b1-5bac6b80d9b2 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:14.653806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:14.654451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:14.654732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:14.655014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:14.656873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:14.718959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:14.720385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:14.729842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1168/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:17.976361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.991859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e015a8c-3d45-49b7-8a28-c05b1e75d5fc service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.998062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:17.998965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.999415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.999996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:18.002059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:18.018002 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:18.019977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:18.036912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1188/2356] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:21.780812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:21.782392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:21.797307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91c3fd-2edf-4043-a15a-26721742c5c5 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:21.802265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:21.803151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:21.803417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:21.803696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] PATH_INFO: `/v3/users/dfc2fed192eb456d87b711d0d44e107f` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:21.804912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc2fed192eb456d87b711d0d44e107f)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:21.806839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:21.854219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc2fed192eb456d87b711d0d44e107f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.861305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc2fed192eb456d87b711d0d44e107f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.868304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc2fed192eb456d87b711d0d44e107f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.868915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc2fed192eb456d87b711d0d44e107f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.869521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc2fed192eb456d87b711d0d44e107f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.887280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1169/2357] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:21 2022] DELETE /identity/v3/users/dfc2fed192eb456d87b711d0d44e107f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:22.083641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.107133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb7edb-9d5a-4919-88d4-947581d5bc18 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.167864 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.168665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.168950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.169224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] PATH_INFO: `/v3/projects/22e65c0cc9524c54881d604ed1291a01` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.170426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e65c0cc9524c54881d604ed1291a01)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.172338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.230246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e65c0cc9524c54881d604ed1291a01', 'request_id': 'req-866fdba4-95f9-4693-93bc-7395e4db5d89'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.238752 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e65c0cc9524c54881d604ed1291a01', 'request_id': 'req-866fdba4-95f9-4693-93bc-7395e4db5d89'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.245723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e65c0cc9524c54881d604ed1291a01', 'request_id': 'req-866fdba4-95f9-4693-93bc-7395e4db5d89'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.257805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] The token cache is being invalidate because project 22e65c0cc9524c54881d604ed1291a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:22.258160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.259417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.261073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1189/2358] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:22 2022] DELETE /identity/v3/projects/22e65c0cc9524c54881d604ed1291a01 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:22.265053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.266544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:22.280267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1607bee-f98a-49a6-9dcf-e078736518ab admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.335042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.335914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.336196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.336473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] PATH_INFO: `/v3/users/cf4cc5c20f714e38aa617db09cf938f0` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.347255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:22.350462 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf4cc5c20f714e38aa617db09cf938f0)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.352729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.401023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf4cc5c20f714e38aa617db09cf938f0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.407910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf4cc5c20f714e38aa617db09cf938f0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.414753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf4cc5c20f714e38aa617db09cf938f0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.415408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf4cc5c20f714e38aa617db09cf938f0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.416016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf4cc5c20f714e38aa617db09cf938f0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.433572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1170/2359] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:22 2022] DELETE /identity/v3/users/cf4cc5c20f714e38aa617db09cf938f0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:22.616979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.618966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:22.634695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fdba4-95f9-4693-93bc-7395e4db5d89 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.698817 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.699589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.700117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.700254 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] PATH_INFO: `/v3/projects/74e7fdc61338416cacf6adea0e5f7c47` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.712334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e7fdc61338416cacf6adea0e5f7c47)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.714307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.725169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:22.767412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e7fdc61338416cacf6adea0e5f7c47', 'request_id': 'req-73b23ec4-a7fd-4e53-82b6-92a541dc6628'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.774164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e7fdc61338416cacf6adea0e5f7c47', 'request_id': 'req-73b23ec4-a7fd-4e53-82b6-92a541dc6628'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.780740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e7fdc61338416cacf6adea0e5f7c47', 'request_id': 'req-73b23ec4-a7fd-4e53-82b6-92a541dc6628'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.793104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] The token cache is being invalidate because project 74e7fdc61338416cacf6adea0e5f7c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:22.793389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.794039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.795573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1190/2360] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:22 2022] DELETE /identity/v3/projects/74e7fdc61338416cacf6adea0e5f7c47 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:24.064313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:24.066109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:24.179367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850f6e0-2946-4a1c-b989-4d9ca7d726a8 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:24.259952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:24.260651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:24.260939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:24.261215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:24.263085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:24.276427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:24.278156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:24.283187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:24.293332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1171/2361] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:32.790055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:34.299781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:13.580225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:13.581672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:13.699449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b23ec4-a7fd-4e53-82b6-92a541dc6628 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:13.705643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:13.706448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:13.706579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:13.706844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:13.708782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:13.722278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:13.723958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:13.733899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1191/2362] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:23.729853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=54 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:12.557316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:12.558730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:12.637724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b66e25-e43b-467f-a09d-65787892078b service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:12.643393 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:12.644038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:12.644324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:12.644636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:12.646428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:12.716599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:12.718175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:12.727324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1172/2363] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:22.115460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:22.116988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:22.207537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef926-c491-480e-a46f-56d7bee707a2 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:22.211769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:22.212414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:22.212729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:22.213007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:22.214834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:22.284301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:22.285786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:22.305076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1192/2364] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:22.723105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=62 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:24.194315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:24.203148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:24.224396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dbf8-8579-4248-a758-e0205aabe37d service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:24.241231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:24.242933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:24.243276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:24.243725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:24.248261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:24.299571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:24.302856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:24.317384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1173/2365] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:27.594503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:27.596016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:27.663833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ad0db-9665-407d-abfd-c5e78d665eb6 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:27.669181 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:27.669839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:27.670120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:27.670395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:27.672206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:27.695370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:27.741013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:27.742446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:27.752371 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1193/2366] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:30.953627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:30.955281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:30.979688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96463f-ddf7-4143-997a-9abe605aca06 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:30.984626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:30.985501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:30.985782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:30.986057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] PATH_INFO: `/v3/users/b97b26dd3bb5427a8edfe4eb653b945b` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:30.987236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97b26dd3bb5427a8edfe4eb653b945b)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:30.989205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.045550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97b26dd3bb5427a8edfe4eb653b945b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.054228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97b26dd3bb5427a8edfe4eb653b945b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.061500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97b26dd3bb5427a8edfe4eb653b945b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.062155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97b26dd3bb5427a8edfe4eb653b945b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.062758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-502b053f-3f2a-4338-a374-64cfebf56b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97b26dd3bb5427a8edfe4eb653b945b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.081106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1174/2367] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:30 2022] DELETE /identity/v3/users/b97b26dd3bb5427a8edfe4eb653b945b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:31.260728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:31.270704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e3db-4159-4da8-ac9d-f5400b52df3e service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:31.335899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:31.336707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.336993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.337266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] PATH_INFO: `/v3/projects/ff102b486fa846498ea8013ca6bcc63c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.338449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff102b486fa846498ea8013ca6bcc63c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:31.340349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.389046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff102b486fa846498ea8013ca6bcc63c', 'request_id': 'req-844d0c1e-31db-446a-a4b2-770baa3ff360'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.397025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff102b486fa846498ea8013ca6bcc63c', 'request_id': 'req-844d0c1e-31db-446a-a4b2-770baa3ff360'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.402686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff102b486fa846498ea8013ca6bcc63c', 'request_id': 'req-844d0c1e-31db-446a-a4b2-770baa3ff360'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.414216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] The token cache is being invalidate because project ff102b486fa846498ea8013ca6bcc63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:31.414509 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.415135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-844d0c1e-31db-446a-a4b2-770baa3ff360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:31.416694 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1194/2368] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:31 2022] DELETE /identity/v3/projects/ff102b486fa846498ea8013ca6bcc63c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:31.423187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49661ba4-9481-4905-94fc-baa4b5dd3e2a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.423492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49661ba4-9481-4905-94fc-baa4b5dd3e2a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.423769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49661ba4-9481-4905-94fc-baa4b5dd3e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.472242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-49661ba4-9481-4905-94fc-baa4b5dd3e2a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:31.473502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-49661ba4-9481-4905-94fc-baa4b5dd3e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:31.536101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1175/2369] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:31.544369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255586a-f39c-4b44-aeb5-83c65da07d40 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.544739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255586a-f39c-4b44-aeb5-83c65da07d40 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.545100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255586a-f39c-4b44-aeb5-83c65da07d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.563187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7255586a-f39c-4b44-aeb5-83c65da07d40 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:31.564381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7255586a-f39c-4b44-aeb5-83c65da07d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:31.572253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1195/2370] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:31.576346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49661ba4-9481-4905-94fc-baa4b5dd3e2a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:31.586457 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49661ba4-9481-4905-94fc-baa4b5dd3e2a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:31.590919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:31.591570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.591848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.592121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.592875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:44:31.593177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:31.594755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.606774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1176/2371] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:44:31.610914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255586a-f39c-4b44-aeb5-83c65da07d40 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:31.620369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255586a-f39c-4b44-aeb5-83c65da07d40 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:31.625081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:31.625726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.626009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.626282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.626982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:31.628888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.664443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1196/2372] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:31 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:31.668692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:31.678815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f4b03-9267-4a1b-b3ec-0258080040fe admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:31.699271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:31.741080 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:31.741788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.742076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.742354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.743064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:31.745024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.770022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1177/2373] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:31 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:31.771760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:31.781989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0525-3fc5-4969-87ea-edfd24798c54 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:31.786755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:31.787403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.787683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.787959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.788714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:31.790292 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.795471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:31.801425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1197/2374] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:31.806119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:31.816091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa392fc3-f62e-4a64-94c9-0308d4c81f4f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:31.821355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62964233-94bd-4890-a3c5-d4149a258d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:31.822014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62964233-94bd-4890-a3c5-d4149a258d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.822299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62964233-94bd-4890-a3c5-d4149a258d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.822579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62964233-94bd-4890-a3c5-d4149a258d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.823303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62964233-94bd-4890-a3c5-d4149a258d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:31.824928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62964233-94bd-4890-a3c5-d4149a258d55 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.837599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1178/2375] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:31.841977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:31.851976 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cb11-1dac-4f61-a8c6-3032c6fbcb4f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:31.856755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:31.857642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.857921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.858194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] PATH_INFO: `/v3/projects/a875b15a4a9f4211bcae0ea6aa2b1d78/users/ff9b475266404f26b37fa018620a6a56/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.878137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a875b15a4a9f4211bcae0ea6aa2b1d78, user_id=ff9b475266404f26b37fa018620a6a56, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:31.881875 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:31.897596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1198/2376] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:31 2022] PUT /identity/v3/projects/a875b15a4a9f4211bcae0ea6aa2b1d78/users/ff9b475266404f26b37fa018620a6a56/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:31.908455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1c0f9-3288-45c9-8d07-8499e3a463bb None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:31.908768 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1c0f9-3288-45c9-8d07-8499e3a463bb None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:31.909047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1c0f9-3288-45c9-8d07-8499e3a463bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:31.934630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d8c1c0f9-3288-45c9-8d07-8499e3a463bb None None] MFA Rules not processed for user `ff9b475266404f26b37fa018620a6a56`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:31.935820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d8c1c0f9-3288-45c9-8d07-8499e3a463bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:31.998110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1179/2377] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:44:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:32.002501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:32.065039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdea0e6-7a94-4143-a4cf-78491255d0a6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:32.070258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:32.070911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:32.071198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:32.071476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:32.072157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:32.074094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:32.099569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1199/2378] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:32 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:32.103716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1c0f9-3288-45c9-8d07-8499e3a463bb None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:32.115087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1c0f9-3288-45c9-8d07-8499e3a463bb None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:32.170429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:32.171099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:32.171389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:32.171670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:32.172362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:32.174314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:32.194788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1180/2379] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:32 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:32.198871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:32.209202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd811-f35c-406f-a82e-d467a4c2e0fa admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:32.213815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:32.214470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:32.214747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:32.215018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:32.215738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:32.217455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:32.228018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1200/2380] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:32.232296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:32.242808 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a6324-e145-40f2-947e-bc38bfe1d423 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:32.247345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:32.248206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:32.248484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:32.248790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] PATH_INFO: `/v3/projects/9bf6dca6faa643a8b137f38169dd61ac/users/be5c009ab26749aa8e0979ca2d1b00b0/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:32.263677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf6dca6faa643a8b137f38169dd61ac, user_id=be5c009ab26749aa8e0979ca2d1b00b0, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:32.266404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:32.274276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:32.280103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1181/2381] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:32 2022] PUT /identity/v3/projects/9bf6dca6faa643a8b137f38169dd61ac/users/be5c009ab26749aa8e0979ca2d1b00b0/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:32.283823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:32.293790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca72ee-b9e3-46c4-8e28-2c6786bc5c69 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:32.350500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b7ba5-5b96-4107-9563-a94ed1436d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:32.351087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b7ba5-5b96-4107-9563-a94ed1436d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:32.351373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b7ba5-5b96-4107-9563-a94ed1436d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:32.351652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b7ba5-5b96-4107-9563-a94ed1436d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:32.352375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b7ba5-5b96-4107-9563-a94ed1436d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:32.354075 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b7ba5-5b96-4107-9563-a94ed1436d74 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:32.363703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1201/2382] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:32.368394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:32.378512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6fc1-a18a-453a-8925-691a4e911f97 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:32.384136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:32.385083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:32.385365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:32.385642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] PATH_INFO: `/v3/projects/9bf6dca6faa643a8b137f38169dd61ac/users/be5c009ab26749aa8e0979ca2d1b00b0/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:32.387873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf6dca6faa643a8b137f38169dd61ac, user_id=be5c009ab26749aa8e0979ca2d1b00b0, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:32.391499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:32.403884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1182/2383] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:32 2022] PUT /identity/v3/projects/9bf6dca6faa643a8b137f38169dd61ac/users/be5c009ab26749aa8e0979ca2d1b00b0/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:32.414066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:32.414414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:32.414701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:32.441092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] MFA Rules not processed for user `be5c009ab26749aa8e0979ca2d1b00b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:32.442295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:32.474989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:32.510436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1202/2384] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:44:32 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:42.399540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=6,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:42.507451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=13 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:12.354782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:12.356550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:12.438105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200187cb-f058-4028-8ef0-04a13414c0fe admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:12.441827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:12.442467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:12.442777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:12.443141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:12.444947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:12.525405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:12.527045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:12.538899 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1183/2385] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:22.533447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=68 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:02.137117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:02.138520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:02.150690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fd62c-fc68-4e89-898b-e8f7a225ea5f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:02.154232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:02.154858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:02.155138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:02.155409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:02.157107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:02.170646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:02.172052 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:02.183011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1203/2386] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:12.178296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:22.528061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:22.529489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:22.586788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba601e8b-d16f-4fe5-969b-f103b3875fbb service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:22.590489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:22.591147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:22.591434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:22.591716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:22.593648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:22.658893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:22.660354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:22.670411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1184/2387] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:22.776053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:22.787576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc7cc4-3057-4f1f-997d-d17576645e78 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:22.791846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:22.792518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:22.792895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:22.793131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:22.794965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:22.872642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:22.874128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:22.885140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1204/2388] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:22.929246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:22.938330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cbf5e-2616-487b-9e94-f931b69ecb57 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:22.942572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:22.943221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:22.943507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:22.943783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:22.954082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:22.955617 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:22.968205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1185/2389] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:22 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:24.459259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:24.460743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:24.475419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e85bb4-5d89-4d9e-a469-4c1ab1c16353 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:24.479456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2fb68-ea76-405d-862b-ab1450455dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:24.480115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2fb68-ea76-405d-862b-ab1450455dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:24.480397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2fb68-ea76-405d-862b-ab1450455dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:24.480705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2fb68-ea76-405d-862b-ab1450455dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:24.541554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2fb68-ea76-405d-862b-ab1450455dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:24.543124 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2fb68-ea76-405d-862b-ab1450455dc3 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:24.554764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1205/2390] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:46:32.963033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=54 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:34.548906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=61 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:18.497900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:18.499413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:18.605779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36e4f4-4b69-4744-878b-c412c0330121 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:18.612506 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:18.613312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:18.613646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:18.613976 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:18.616396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:18.632788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:18.634547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:18.646370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1186/2391] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:25.897532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:25.898457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:25.898457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:25.933008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] MFA Rules not processed for user `f8333f6a9dab4618bc0e6b2060f15250`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:25.935015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:26.016252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1206/2392] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:48:26.432759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:26.448375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fa54-ec46-4293-8838-bdf0198283be service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:26.454928 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:26.455919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:26.456205 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:26.456599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:26.459111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:26.473026 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:26.475321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:26.485009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1187/2393] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:26.913839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:26.916429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:26.935107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1100-4a66-4a33-a502-a98b88e5ab05 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:26.942600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:26.944408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:26.944740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:26.945027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:26.947036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:26.961076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:26.963014 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:26.976016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1207/2394] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:48:29.159358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:29.169979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf2af2-93b6-492e-9714-e79dded947a1 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:29.175438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:29.176140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:29.176452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:29.176771 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:29.179553 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:29.193324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:29.194794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:29.206502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1188/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:36.824156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:36.829123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:36.863821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0aba20-16e6-4164-b970-ff382c627c27 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:36.877433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:36.879152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:36.879649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:36.879981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:36.882078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:36.896050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:36.897821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:36.911399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1208/2396] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:39.199937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=49 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:40.772167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:40.795224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affa1dc-7571-4c9c-af47-56b1dacda1ed service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:40.805088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:40.806100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:40.806573 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:40.807027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:40.810588 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:40.827001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:40.829184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:40.841812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1189/2397] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:46.904826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=40 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:50.836423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:07.551355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:07.553405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:07.578006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289af6-53dc-443d-a9fa-fcfe70b35246 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:07.586395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:07.587392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:07.587810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:07.588229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:07.590988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:07.627471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:07.630533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:07.663828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1209/2398] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:50:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:50:17.660815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=14 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:23.646769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:23.658693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38663e0-9dde-4fd0-be0e-1247a783f4af service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:23.662877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7232133-ffae-4d8d-8d36-51758ccde7bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:23.663527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7232133-ffae-4d8d-8d36-51758ccde7bc service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:23.663809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7232133-ffae-4d8d-8d36-51758ccde7bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:23.664087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7232133-ffae-4d8d-8d36-51758ccde7bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:23.665951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b7232133-ffae-4d8d-8d36-51758ccde7bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:23.771180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7232133-ffae-4d8d-8d36-51758ccde7bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:23.773733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7232133-ffae-4d8d-8d36-51758ccde7bc service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:23.790221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1190/2399] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:50:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:50:24.759980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:24.884441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a922-4735-4ffd-968b-3c14fd8b2913 service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:24.891184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:24.892503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:24.892894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:24.893237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] PATH_INFO: `/v3/users/ff9b475266404f26b37fa018620a6a56` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:24.894896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9b475266404f26b37fa018620a6a56)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:24.897188 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:24.966996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9b475266404f26b37fa018620a6a56'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:24.979834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9b475266404f26b37fa018620a6a56'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:24.995356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9b475266404f26b37fa018620a6a56'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:24.996897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9b475266404f26b37fa018620a6a56'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:24.998258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9b475266404f26b37fa018620a6a56'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:25.040419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1210/2400] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:50:24 2022] DELETE /identity/v3/users/ff9b475266404f26b37fa018620a6a56 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:50:25.042342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:25.043225 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:25.044035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:25.110002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:50:25.111301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:25.220803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1191/2401] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:50:25.249111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:25.249111 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:25.347342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:25.350340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:25.400124 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73813080-a983-4fa2-a500-7856594242c9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:25.407695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:25.409008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:25.409528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:25.410279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:25.432723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:25.434708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:25.449941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1211/2402] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:50:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:50:25.487081 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b304849-8548-4b3d-bca7-65f9ded945d2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:25.493751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:25.494684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:25.495095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:25.495487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:25.498689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:25.513374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:25.515100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:25.530294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1192/2403] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:50:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:50:26.014208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:26.015715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:26.032835 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:26.041073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadea1a-788c-4d6b-80d7-fc8b340404ec service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:26.047375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:26.048340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:26.049018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:26.049758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] PATH_INFO: `/v3/projects/a875b15a4a9f4211bcae0ea6aa2b1d78` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:26.053489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a875b15a4a9f4211bcae0ea6aa2b1d78)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:26.061034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:26.145458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a875b15a4a9f4211bcae0ea6aa2b1d78', 'request_id': 'req-75967da4-98e6-4ecc-ad76-b8ebc34163e3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.158127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a875b15a4a9f4211bcae0ea6aa2b1d78', 'request_id': 'req-75967da4-98e6-4ecc-ad76-b8ebc34163e3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.167996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a875b15a4a9f4211bcae0ea6aa2b1d78', 'request_id': 'req-75967da4-98e6-4ecc-ad76-b8ebc34163e3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.198563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] The token cache is being invalidate because project a875b15a4a9f4211bcae0ea6aa2b1d78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:50:26.198948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.199545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.201919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1212/2404] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:50:26 2022] DELETE /identity/v3/projects/a875b15a4a9f4211bcae0ea6aa2b1d78 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:50:26.206392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:26.212815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:26.221326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60137-87a7-4af0-999d-b9d7fb8c3210 service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:26.317276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:26.318745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:26.319259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:26.319739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] PATH_INFO: `/v3/users/be5c009ab26749aa8e0979ca2d1b00b0` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:26.321978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5c009ab26749aa8e0979ca2d1b00b0)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:26.324283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:26.397025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5c009ab26749aa8e0979ca2d1b00b0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.407994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5c009ab26749aa8e0979ca2d1b00b0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.422810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5c009ab26749aa8e0979ca2d1b00b0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.424123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5c009ab26749aa8e0979ca2d1b00b0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.425384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-6b32a4e8-1806-4d03-9cc2-bc34ab6a4366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5c009ab26749aa8e0979ca2d1b00b0'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:26.447164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1193/2405] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:50:26 2022] DELETE /identity/v3/users/be5c009ab26749aa8e0979ca2d1b00b0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:50:26.820947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:26.823238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:26.858074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75967da4-98e6-4ecc-ad76-b8ebc34163e3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:26.935416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:26.936902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:26.937378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:26.937872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] PATH_INFO: `/v3/projects/9bf6dca6faa643a8b137f38169dd61ac` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:26.940020 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf6dca6faa643a8b137f38169dd61ac)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:26.943447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:26.997799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf6dca6faa643a8b137f38169dd61ac', 'request_id': 'req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:27.006707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf6dca6faa643a8b137f38169dd61ac', 'request_id': 'req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:27.013638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf6dca6faa643a8b137f38169dd61ac', 'request_id': 'req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:27.030786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] The token cache is being invalidate because project 9bf6dca6faa643a8b137f38169dd61ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:50:27.031904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:27.032297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-6122f4a8-3c76-4cd7-be18-f4a3a0292d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:50:27.034409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1213/2406] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:50:26 2022] DELETE /identity/v3/projects/9bf6dca6faa643a8b137f38169dd61ac => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:50:27.039572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee78cd-1a38-44d2-b657-76a8f655f161 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.039876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee78cd-1a38-44d2-b657-76a8f655f161 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.040173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee78cd-1a38-44d2-b657-76a8f655f161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.071795 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-bfee78cd-1a38-44d2-b657-76a8f655f161 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:50:27.073038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bfee78cd-1a38-44d2-b657-76a8f655f161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:27.153423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1194/2407] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:50:27.163867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21776e-83da-4943-9385-55bc8e82dea5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.164211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21776e-83da-4943-9385-55bc8e82dea5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.164494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21776e-83da-4943-9385-55bc8e82dea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.186194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6e21776e-83da-4943-9385-55bc8e82dea5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:50:27.187613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6e21776e-83da-4943-9385-55bc8e82dea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:27.196431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1214/2408] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:50:27.202559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee78cd-1a38-44d2-b657-76a8f655f161 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.220040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee78cd-1a38-44d2-b657-76a8f655f161 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.227907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.228632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.229146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.229425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.230579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:50:27.231089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.233989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.249499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:27.255719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1195/2409] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:50:27.261430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21776e-83da-4943-9385-55bc8e82dea5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.278837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21776e-83da-4943-9385-55bc8e82dea5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.283750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.284481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.284797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.285073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.285769 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.287746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.296747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:27.326310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1215/2410] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:50:27 2022] POST /identity/v3/projects => generated 391 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:50:27.331578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.346255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64f860-4459-4709-9df7-412886ac6ec6 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.430767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.431810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.432307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.432823 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.434208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.437934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.463696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1196/2411] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:50:27 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:50:27.470602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.481234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0852eb-b444-4621-be80-446f37aa9b2c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.488059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.489125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.489631 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.490090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.491380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.495161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.509990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1216/2412] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:50:27.515158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.526206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53745c-6700-4ac9-8d11-4bdd1304b246 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.531630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e169b23-b9e6-423f-8bc1-5db8979b556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.532300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169b23-b9e6-423f-8bc1-5db8979b556d admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.532636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169b23-b9e6-423f-8bc1-5db8979b556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.532912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169b23-b9e6-423f-8bc1-5db8979b556d admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.533646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e169b23-b9e6-423f-8bc1-5db8979b556d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.535238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e169b23-b9e6-423f-8bc1-5db8979b556d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.551450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1197/2413] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:50:27.555825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.567386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85316110-1782-4646-8b96-dc059e736648 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.574824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.576878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.577364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.577790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] PATH_INFO: `/v3/projects/b071758c681945a0b94807462a37e4b5/users/3b1900ca6c45426382e25dfbad5e73f8/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.599468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b071758c681945a0b94807462a37e4b5, user_id=3b1900ca6c45426382e25dfbad5e73f8, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.602295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.617387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1217/2414] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:50:27 2022] PUT /identity/v3/projects/b071758c681945a0b94807462a37e4b5/users/3b1900ca6c45426382e25dfbad5e73f8/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:50:27.631336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7eec4-0137-4f23-8ee2-40db0e1c0587 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.631812 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7eec4-0137-4f23-8ee2-40db0e1c0587 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.632263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7eec4-0137-4f23-8ee2-40db0e1c0587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.678611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f5c7eec4-0137-4f23-8ee2-40db0e1c0587 None None] MFA Rules not processed for user `3b1900ca6c45426382e25dfbad5e73f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:50:27.680539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f5c7eec4-0137-4f23-8ee2-40db0e1c0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:27.758015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1198/2415] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:50:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:50:27.763095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.829911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217741b3-0e2b-4dfd-b507-fbbb8fcf78a5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.834874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.835563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.835819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.836091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.836834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.838753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.867221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1218/2416] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:50:27 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:50:27.874169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7eec4-0137-4f23-8ee2-40db0e1c0587 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:27.893342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7eec4-0137-4f23-8ee2-40db0e1c0587 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:27.974539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:27.975228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:27.975513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:27.975786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:27.976501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:27.978460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:27.995179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:28.005712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1199/2417] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:50:27 2022] POST /identity/v3/users => generated 303 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:50:28.007441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:28.025213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9ed4-edf6-46e9-a08e-c89683911cdf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:28.030933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:28.031701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:28.031906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:28.032184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:28.032968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:28.034602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:28.047376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1219/2418] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:50:28.052094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:28.066150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81acdfd-4737-43ae-9f8a-ca0e891d3fcc admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:28.073679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:28.075097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:28.075486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:28.075962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] PATH_INFO: `/v3/projects/d5b00d162e68421dab81b2adb4286461/users/7d0192e318504785a8ad58b6292793a9/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:28.097732 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b00d162e68421dab81b2adb4286461, user_id=7d0192e318504785a8ad58b6292793a9, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:28.101432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:28.121447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1200/2419] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:50:28 2022] PUT /identity/v3/projects/d5b00d162e68421dab81b2adb4286461/users/7d0192e318504785a8ad58b6292793a9/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:50:28.127060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:28.139744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea256f-6953-4ec1-9a66-a91b980378ff admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:28.208365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2f17f-2fbf-4e00-a375-da80fcaef8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:28.210506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f17f-2fbf-4e00-a375-da80fcaef8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:28.212024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f17f-2fbf-4e00-a375-da80fcaef8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:28.215259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f17f-2fbf-4e00-a375-da80fcaef8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:28.218197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2f17f-2fbf-4e00-a375-da80fcaef8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:28.221416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2f17f-2fbf-4e00-a375-da80fcaef8fb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:28.238914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1220/2420] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:50:28.243403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:28.255116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c36d9-925e-419a-95e3-ae6b8de3d552 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:28.260136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:28.261074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:28.261353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:28.261629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] PATH_INFO: `/v3/projects/d5b00d162e68421dab81b2adb4286461/users/7d0192e318504785a8ad58b6292793a9/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:28.264255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b00d162e68421dab81b2adb4286461, user_id=7d0192e318504785a8ad58b6292793a9, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:28.266981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:28.280706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1201/2421] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:50:28 2022] PUT /identity/v3/projects/d5b00d162e68421dab81b2adb4286461/users/7d0192e318504785a8ad58b6292793a9/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:50:28.294775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:28.295229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:28.295744 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:28.340639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] MFA Rules not processed for user `7d0192e318504785a8ad58b6292793a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:50:28.341873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:28.354454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:28.431714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1221/2422] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:50:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:50:38.275435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=16,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:38.428313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=23 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:25.398613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:25.401183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:25.442763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:25.445107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:25.487192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68b301-6b62-4ac7-a064-5cf9da5cb23e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:25.491032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:25.491685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:25.491972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:25.492242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:25.494125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:25.522886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf6095-d925-408b-b018-ccd36c824bc4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:25.527479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:25.527979 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:25.528262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:25.528539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:25.530410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:25.608283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:25.608529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:25.609821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:25.610885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:25.639133 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1202/2423] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:52:25.645234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1222/2424] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:52:25.703261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:25.721475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47856f-f13d-443a-9ee1-b415d15d2906 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:25.725790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:25.726836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:25.727528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:25.727528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:25.749314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:25.750877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:25.767772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1203/2425] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:25.964333 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:25.968514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:25.999162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:26.015466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2c651-2089-44ed-b523-97799929a9be service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:26.026296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:26.026296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:26.031188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:26.031188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:26.070391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf01a6-1712-43e4-a06f-fd13d80dd1e0 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:26.074971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:26.075878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:26.076374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:26.076890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:26.079211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:26.124984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:26.126929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:26.140332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1204/2426] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:52:26.143123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:26.162953 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:26.164828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:26.185473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1223/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:29.671322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:29.673612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:29.690886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:29.760430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:29.764727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:29.788702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4e392-5623-4048-831a-dd218005110f service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:29.796730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a5384-79b2-4132-8865-b12b83319fe3 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:29.797085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:29.797850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:29.798140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:29.798448 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:29.801725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:29.820206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:29.821747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:29.822804 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:29.823301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] PATH_INFO: `/v3/users/6ea7013dd9424b9db1798e81521ee903` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:29.823431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:29.825740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea7013dd9424b9db1798e81521ee903)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:29.827900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:29.828049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:29.849563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1205/2428] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:29.932225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea7013dd9424b9db1798e81521ee903'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:29.945433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea7013dd9424b9db1798e81521ee903'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:29.954199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea7013dd9424b9db1798e81521ee903'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:29.955790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea7013dd9424b9db1798e81521ee903'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:29.956416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f4b8d5cc-8a1d-4019-966c-c28a55192007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea7013dd9424b9db1798e81521ee903'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:29.983255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1224/2429] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:52:29 2022] DELETE /identity/v3/users/6ea7013dd9424b9db1798e81521ee903 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:52:30.179777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:30.206183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47310f80-44b4-4b22-bd7c-6d497250c81c service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:30.301945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:30.302829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.303111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.303385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] PATH_INFO: `/v3/projects/eec4b7af6b284140be130acf01e1710f` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.304606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec4b7af6b284140be130acf01e1710f)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:30.306526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:30.358915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec4b7af6b284140be130acf01e1710f', 'request_id': 'req-71ddb854-2e5f-41d1-8cde-92d000789102'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:30.367266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec4b7af6b284140be130acf01e1710f', 'request_id': 'req-71ddb854-2e5f-41d1-8cde-92d000789102'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:30.373852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec4b7af6b284140be130acf01e1710f', 'request_id': 'req-71ddb854-2e5f-41d1-8cde-92d000789102'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:30.385659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] The token cache is being invalidate because project eec4b7af6b284140be130acf01e1710f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:52:30.385994 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:30.387203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-71ddb854-2e5f-41d1-8cde-92d000789102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:52:30.388797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1206/2430] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:52:30 2022] DELETE /identity/v3/projects/eec4b7af6b284140be130acf01e1710f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:52:30.394743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ba8a-6658-4397-961c-39fed41d6dd9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.395027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ba8a-6658-4397-961c-39fed41d6dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.395428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ba8a-6658-4397-961c-39fed41d6dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.414711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7648ba8a-6658-4397-961c-39fed41d6dd9 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:52:30.416048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7648ba8a-6658-4397-961c-39fed41d6dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:30.494940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1225/2431] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:52:30.503921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254cef7-f942-4d8a-8f42-062e78e21d30 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.504206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254cef7-f942-4d8a-8f42-062e78e21d30 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.504546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254cef7-f942-4d8a-8f42-062e78e21d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.523959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3254cef7-f942-4d8a-8f42-062e78e21d30 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:52:30.525214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3254cef7-f942-4d8a-8f42-062e78e21d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:30.533824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1207/2432] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:52:30.538343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648ba8a-6658-4397-961c-39fed41d6dd9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:30.550412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648ba8a-6658-4397-961c-39fed41d6dd9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:30.555329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:30.556119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.556487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.556898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.557735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:52:30.558145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:30.560414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:30.572823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1226/2433] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:52:30.577901 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254cef7-f942-4d8a-8f42-062e78e21d30 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:30.588631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254cef7-f942-4d8a-8f42-062e78e21d30 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:30.593498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:30.594202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.594466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.594743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.595451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:30.597431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:30.639881 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1208/2434] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:52:30 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:52:30.641496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:30.652408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa85952-fb0c-41b4-afc3-563dbe3ae4ad admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:30.710750 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:30.726206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:30.726909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.727190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.727467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.728160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:30.730144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:30.752829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1227/2435] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:52:30 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:52:30.757028 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:30.763435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:30.770052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bc831-fa86-4ee0-af76-a9046df8191d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:30.775790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:30.776844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.777243 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.777525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.778248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:30.779830 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:30.790491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1209/2436] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:52:30.794589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:30.805065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05764f11-9cdc-40d8-b248-f95fd4163fcf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:30.810403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc07226-c835-43a0-9a7c-43057264749d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:30.811077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc07226-c835-43a0-9a7c-43057264749d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.811363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc07226-c835-43a0-9a7c-43057264749d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.811642 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc07226-c835-43a0-9a7c-43057264749d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.812361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc07226-c835-43a0-9a7c-43057264749d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:30.814000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc07226-c835-43a0-9a7c-43057264749d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:30.825355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1228/2437] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:52:30.829176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:30.840335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662c802-ead8-4a5a-afc9-c3e901553e4f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:30.845602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:30.846499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.846782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.847145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] PATH_INFO: `/v3/projects/77479cd1032542df98070a598e04b824/users/12276fc1f9fb43d8aefb37412f6d6545/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.864329 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77479cd1032542df98070a598e04b824, user_id=12276fc1f9fb43d8aefb37412f6d6545, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:30.867068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:30.884003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1210/2438] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:52:30 2022] PUT /identity/v3/projects/77479cd1032542df98070a598e04b824/users/12276fc1f9fb43d8aefb37412f6d6545/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:52:30.896634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:30.896952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:30.897232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:30.930753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] MFA Rules not processed for user `12276fc1f9fb43d8aefb37412f6d6545`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:52:30.932914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:31.027055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1229/2439] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:52:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:52:31.039267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:31.041359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:31.117557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8981bbe-3683-440a-9d1b-b8b96641b4c2 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:31.123803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:31.124458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:31.125952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:31.125952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:31.134185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:31.135785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:31.145086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1211/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:31.166186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:31.167581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:31.225896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486fcd9-b7e3-4aae-b323-0fa913ae3e01 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:31.229527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:31.230674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:31.230958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:31.231231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:31.233036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:31.252480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:31.253971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:31.265856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1230/2441] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:31.649751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:31.659321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3caef-249c-4d21-8181-f46ea23a145f service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:31.663533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3808113-0864-4622-858e-8f01c85074ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:31.664173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808113-0864-4622-858e-8f01c85074ee service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:31.664449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808113-0864-4622-858e-8f01c85074ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:31.664757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808113-0864-4622-858e-8f01c85074ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:31.666522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b3808113-0864-4622-858e-8f01c85074ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:31.681517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808113-0864-4622-858e-8f01c85074ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:31.683017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808113-0864-4622-858e-8f01c85074ee service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:31.694963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1212/2442] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:41.260619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:41.690029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:26.471116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:26.472543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:26.557581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caf6e5-72bd-4797-80a6-a76b9c89c3d2 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:26.562957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:26.563635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:26.563920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:26.564203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:26.566050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:26.578507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:26.580087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:26.590917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1231/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:36.586230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=36 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:26.762821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f2bb4-0ea9-4e0c-a85e-ab26884797fd None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:26.763386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f2bb4-0ea9-4e0c-a85e-ab26884797fd None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:26.763546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f2bb4-0ea9-4e0c-a85e-ab26884797fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:26.825159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a59f2bb4-0ea9-4e0c-a85e-ab26884797fd None None] MFA Rules not processed for user `8b60a3f963f34bbab49682c9d7a80bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:26.826376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a59f2bb4-0ea9-4e0c-a85e-ab26884797fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:26.930187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1213/2444] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:27.173837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:27.175489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:27.266219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c67b-6f4d-43c5-8648-cae12fa7c9c4 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:27.269914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:27.270562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:27.270844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:27.271121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:27.272964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:27.283414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:27.284867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:27.293209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1232/2445] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:27.641616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f2bb4-0ea9-4e0c-a85e-ab26884797fd None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:27.671505 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f2bb4-0ea9-4e0c-a85e-ab26884797fd None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:27.677932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:27.679160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:27.679160 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:27.679376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:27.681044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:27.694143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:27.695776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:27.708975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1214/2446] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:29.184352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:29.193638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16976e4b-812e-400c-8f2c-971ffa4de10b service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:29.197492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:29.198214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:29.198502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:29.198780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:29.200983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:29.212987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:29.214518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:29.223850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1233/2447] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:31.146977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:31.158473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bf5e-345f-48e8-a4a5-883ea5cc25ae service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:31.163186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb5d3-ca92-46a0-9ddb-c5e5a4316ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:31.163850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb5d3-ca92-46a0-9ddb-c5e5a4316ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:31.164155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb5d3-ca92-46a0-9ddb-c5e5a4316ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:31.164778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb5d3-ca92-46a0-9ddb-c5e5a4316ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:31.167364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5fcbb5d3-ca92-46a0-9ddb-c5e5a4316ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:31.280608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbb5d3-ca92-46a0-9ddb-c5e5a4316ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:31.282253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbb5d3-ca92-46a0-9ddb-c5e5a4316ed8 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:31.299257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1215/2448] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:32.300410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:32.309164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165c943-bf56-49b9-a8fb-68f1c0c19528 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:32.314531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:32.315424 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:32.315717 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:32.316339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] PATH_INFO: `/v3/users/12276fc1f9fb43d8aefb37412f6d6545` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:32.317391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=12276fc1f9fb43d8aefb37412f6d6545)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:32.319650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:32.394238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12276fc1f9fb43d8aefb37412f6d6545'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.408417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12276fc1f9fb43d8aefb37412f6d6545'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.423692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12276fc1f9fb43d8aefb37412f6d6545'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.424837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12276fc1f9fb43d8aefb37412f6d6545'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.426015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12276fc1f9fb43d8aefb37412f6d6545'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.455697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1234/2449] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:32 2022] DELETE /identity/v3/users/12276fc1f9fb43d8aefb37412f6d6545 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:32.460611 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:32.461729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:32.462148 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:32.511857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:32.513932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:32.569304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:32.570042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:32.573549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:32.627901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:32.676490 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4683e-d7da-49fd-a506-b3ec93f820ed admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:32.683987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1216/2450] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:32.685863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:32.686878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:32.687235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:32.687587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] PATH_INFO: `/v3/users/3b1900ca6c45426382e25dfbad5e73f8` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:32.689050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b1900ca6c45426382e25dfbad5e73f8)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:32.691101 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:32.693277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:32.695123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:32.793513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b1900ca6c45426382e25dfbad5e73f8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.803990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72a208-d9ea-4273-849a-2e0e7c205093 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:32.804825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b1900ca6c45426382e25dfbad5e73f8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.808266 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dfc0f-ecd3-4ffd-a448-165631edaa0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:32.808973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfc0f-ecd3-4ffd-a448-165631edaa0b service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:32.809287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfc0f-ecd3-4ffd-a448-165631edaa0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:32.810238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfc0f-ecd3-4ffd-a448-165631edaa0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:32.818162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b1900ca6c45426382e25dfbad5e73f8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.818860 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b1900ca6c45426382e25dfbad5e73f8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.820297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b1900ca6c45426382e25dfbad5e73f8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:32.826954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dfc0f-ecd3-4ffd-a448-165631edaa0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:32.831391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dfc0f-ecd3-4ffd-a448-165631edaa0b service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:32.854860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1217/2451] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:54:32.861257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1235/2452] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:32 2022] DELETE /identity/v3/users/3b1900ca6c45426382e25dfbad5e73f8 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:32.877342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:32.877342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:32.877672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:32.902456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:32.904036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:33.022103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1218/2453] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:33.062743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:33.064458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:33.207767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad14e1-4a68-41ba-ac0f-31a4fb86fe3f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:33.213798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105aed7f-4fa6-4da7-b21e-6b3672381372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:33.214479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aed7f-4fa6-4da7-b21e-6b3672381372 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:33.214759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aed7f-4fa6-4da7-b21e-6b3672381372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:33.215031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aed7f-4fa6-4da7-b21e-6b3672381372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:33.235030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105aed7f-4fa6-4da7-b21e-6b3672381372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:33.237047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105aed7f-4fa6-4da7-b21e-6b3672381372 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:33.247896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1236/2454] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:33.508862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:33.510631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:33.536162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5a3fe-83bb-4cc6-91bd-9d85cc56c435 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:33.550729 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:33.551619 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:33.551902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:33.552177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] PATH_INFO: `/v3/projects/77479cd1032542df98070a598e04b824` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:33.555660 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=77479cd1032542df98070a598e04b824)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:33.557797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:33.687662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77479cd1032542df98070a598e04b824', 'request_id': 'req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:33.712355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77479cd1032542df98070a598e04b824', 'request_id': 'req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:33.719528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:33.722860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77479cd1032542df98070a598e04b824', 'request_id': 'req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:33.772375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] The token cache is being invalidate because project 77479cd1032542df98070a598e04b824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:33.772886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:33.773718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a0b8b9c5-8777-4084-90d3-c1340a41d4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:33.777065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1219/2455] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:33 2022] DELETE /identity/v3/projects/77479cd1032542df98070a598e04b824 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:33.782793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc9ff-8083-45e1-ac80-631aceb72da4 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:33.783309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc9ff-8083-45e1-ac80-631aceb72da4 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:33.783402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc9ff-8083-45e1-ac80-631aceb72da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:33.835222 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-57afc9ff-8083-45e1-ac80-631aceb72da4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:33.838307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-57afc9ff-8083-45e1-ac80-631aceb72da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:33.882648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:33.950290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1237/2456] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:33.958279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3eb93-ee95-46d5-bf0d-70789e73b43e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:33.958643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3eb93-ee95-46d5-bf0d-70789e73b43e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:33.959408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3eb93-ee95-46d5-bf0d-70789e73b43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:33.982846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5ad3eb93-ee95-46d5-bf0d-70789e73b43e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:33.985509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5ad3eb93-ee95-46d5-bf0d-70789e73b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:33.995873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1220/2457] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:34.011735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afc9ff-8083-45e1-ac80-631aceb72da4 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.022526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afc9ff-8083-45e1-ac80-631aceb72da4 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.027937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.028658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.028935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.029214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.030350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:54:34.030503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.032322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.048797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1238/2458] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:54:34.054062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3eb93-ee95-46d5-bf0d-70789e73b43e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.066224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3eb93-ee95-46d5-bf0d-70789e73b43e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.073061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.075242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.075643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.076046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.076947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.079964 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.138061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.140739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:34.152086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1221/2459] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:54:34 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:34.153185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.166428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fe460-bfa6-4c44-b339-d70a55ccb8cb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.309708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbfc5d-90ce-4525-9ef9-6064731d6179 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.315564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.316279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.317087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.317372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.317606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.317838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.318024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] PATH_INFO: `/v3/projects/b071758c681945a0b94807462a37e4b5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.318341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.319512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.320683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b071758c681945a0b94807462a37e4b5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.322720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.325339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.354627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1222/2460] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:54:34 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:54:34.358691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.378783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa94aeb-4426-452d-ad6f-8adf01738e2e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.387891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.388635 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.388925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.389209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.391091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.392756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.412749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1223/2461] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:34.415422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.430842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc11263-bd06-4851-a97c-3f33761f16b3 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.455507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b071758c681945a0b94807462a37e4b5', 'request_id': 'req-12d233b8-3841-4f0b-8af1-ac9e925e21d3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.485044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b071758c681945a0b94807462a37e4b5', 'request_id': 'req-12d233b8-3841-4f0b-8af1-ac9e925e21d3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.512314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b071758c681945a0b94807462a37e4b5', 'request_id': 'req-12d233b8-3841-4f0b-8af1-ac9e925e21d3'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.531686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.532421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.532743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.533546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.533909 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.536024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.547000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] The token cache is being invalidate because project b071758c681945a0b94807462a37e4b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:34.548954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.551471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.552110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1224/2462] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:34.556911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.560937 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1239/2463] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:34 2022] DELETE /identity/v3/projects/b071758c681945a0b94807462a37e4b5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:34.563383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.565094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:34.567307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:34.593689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed31d57-3949-429c-b629-e0b6fbba487f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.595350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d233b8-3841-4f0b-8af1-ac9e925e21d3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:34.599537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f89fe4-72b3-4b39-841b-f37d21430727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.600547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f89fe4-72b3-4b39-841b-f37d21430727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.600906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f89fe4-72b3-4b39-841b-f37d21430727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.601439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:34.602314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f89fe4-72b3-4b39-841b-f37d21430727 admin admin] PATH_INFO: `/v3/projects/1d08c10f0feb4953a01ae37166636ed3/users/f67af7f314804e4d9d7cdccfb36f26e7/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.603367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.603868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.604969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] PATH_INFO: `/v3/users/7d0192e318504785a8ad58b6292793a9` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.608010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d0192e318504785a8ad58b6292793a9)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.610845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.626453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f89fe4-72b3-4b39-841b-f37d21430727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d08c10f0feb4953a01ae37166636ed3, user_id=f67af7f314804e4d9d7cdccfb36f26e7, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:34.629888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f89fe4-72b3-4b39-841b-f37d21430727 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:34.653657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1225/2464] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:34 2022] PUT /identity/v3/projects/1d08c10f0feb4953a01ae37166636ed3/users/f67af7f314804e4d9d7cdccfb36f26e7/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:34.664105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:34.664105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:34.665806 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:34.676377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:34.711934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] MFA Rules not processed for user `f67af7f314804e4d9d7cdccfb36f26e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:34.715036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:34.715304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d0192e318504785a8ad58b6292793a9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.721348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:34.729512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d0192e318504785a8ad58b6292793a9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.753965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d0192e318504785a8ad58b6292793a9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.755899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d0192e318504785a8ad58b6292793a9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.757076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d0192e318504785a8ad58b6292793a9'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:34.799361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1240/2465] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:34 2022] DELETE /identity/v3/users/7d0192e318504785a8ad58b6292793a9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:34.851719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1226/2466] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:34.863721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:34.865712 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:34.990247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced9649-f615-4b5f-8174-63224dee50c2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:35.001723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7fb85-afae-4bfd-9e0b-b53172a72c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:35.003168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7fb85-afae-4bfd-9e0b-b53172a72c8c service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:35.003973 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7fb85-afae-4bfd-9e0b-b53172a72c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:35.004520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7fb85-afae-4bfd-9e0b-b53172a72c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:35.142064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7fb85-afae-4bfd-9e0b-b53172a72c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:35.145505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7fb85-afae-4bfd-9e0b-b53172a72c8c service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:35.164842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1241/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:35.239707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:35.241272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:35.387248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e23f4-2b35-47db-a89c-a27207062e7d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:35.564173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:35.565306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:35.565592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:35.566591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] PATH_INFO: `/v3/projects/d5b00d162e68421dab81b2adb4286461` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:35.567643 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b00d162e68421dab81b2adb4286461)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:35.569887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:35.665292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:35.678859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b00d162e68421dab81b2adb4286461', 'request_id': 'req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:35.699501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b00d162e68421dab81b2adb4286461', 'request_id': 'req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:35.709098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b00d162e68421dab81b2adb4286461', 'request_id': 'req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:35.738452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] The token cache is being invalidate because project d5b00d162e68421dab81b2adb4286461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:35.738766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:35.739950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c75fa5d4-cf4c-4fdb-84e5-8a8d27a47103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:35.742187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1227/2468] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:35 2022] DELETE /identity/v3/projects/d5b00d162e68421dab81b2adb4286461 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:35.747627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a32cc-c399-44d7-97d0-04f859198632 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:35.747918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a32cc-c399-44d7-97d0-04f859198632 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:35.748382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a32cc-c399-44d7-97d0-04f859198632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:35.783736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-386a32cc-c399-44d7-97d0-04f859198632 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:35.785041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-386a32cc-c399-44d7-97d0-04f859198632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:35.872272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1242/2469] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:35.882409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc821-00e0-47c4-a767-fd4414de36bf None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:35.882910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc821-00e0-47c4-a767-fd4414de36bf None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:35.883204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc821-00e0-47c4-a767-fd4414de36bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:35.905497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-808fc821-00e0-47c4-a767-fd4414de36bf None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:35.906824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-808fc821-00e0-47c4-a767-fd4414de36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:35.916066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1228/2470] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:54:35.920678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a32cc-c399-44d7-97d0-04f859198632 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:35.927594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:35.931994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a32cc-c399-44d7-97d0-04f859198632 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:35.938406 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:35.939551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:35.939899 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:35.940214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:35.941046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:54:35.941351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:35.943054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:35.960208 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1243/2471] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:54:35.961452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc821-00e0-47c4-a767-fd4414de36bf None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:35.972313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc821-00e0-47c4-a767-fd4414de36bf None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:35.976871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:35.977534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:35.977813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:35.978085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:35.979014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:35.981169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.015727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1229/2472] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:54:35 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:36.023064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.035338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35acc7-67d1-4a8d-a4a4-59c2960819fd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.042580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.045154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.152404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f77d-6615-40cf-8c84-eefec545ed22 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.158182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.159057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.159369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.159640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] PATH_INFO: `/v3/users/00556257371a41b39bce3ca8ee545238` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.160867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=00556257371a41b39bce3ca8ee545238)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.162781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.163055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.163725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.164013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.164303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.165017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.166932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.191543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1244/2473] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:54:36 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:54:36.193519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.204255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b870b-de33-4d32-a881-58464ad67fd2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.209655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.210321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.210606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.210894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.211626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.213268 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.225892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1245/2474] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:36.231042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.241364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00556257371a41b39bce3ca8ee545238'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.242888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e006760-8169-41a7-9ed3-ca32f6074a3c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.247633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.248315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.248777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.249096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.249922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.251803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.252048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00556257371a41b39bce3ca8ee545238'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.260886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00556257371a41b39bce3ca8ee545238'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.261634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00556257371a41b39bce3ca8ee545238'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.262253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e7c70f82-b2b8-431d-955e-eb201ef207ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00556257371a41b39bce3ca8ee545238'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.262795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1246/2475] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:54:36.267516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.271761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.282992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1230/2476] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:36 2022] DELETE /identity/v3/users/00556257371a41b39bce3ca8ee545238 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:36.285007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a189c-abba-4dcc-b6d6-e93b1164df84 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.367948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.368834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.369079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.369355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] PATH_INFO: `/v3/projects/9cdd89402d044366ad745bccaa0109bf/users/178a42a6e05240bd9343df3b3e380eaa/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.390071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdd89402d044366ad745bccaa0109bf, user_id=178a42a6e05240bd9343df3b3e380eaa, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.392803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.407675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1247/2477] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:36 2022] PUT /identity/v3/projects/9cdd89402d044366ad745bccaa0109bf/users/178a42a6e05240bd9343df3b3e380eaa/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:36.418135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d42bc-cb3e-4e7c-b729-6dea600ce8e1 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.418424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d42bc-cb3e-4e7c-b729-6dea600ce8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.418701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d42bc-cb3e-4e7c-b729-6dea600ce8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.447954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f48d42bc-cb3e-4e7c-b729-6dea600ce8e1 None None] MFA Rules not processed for user `178a42a6e05240bd9343df3b3e380eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:36.449204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f48d42bc-cb3e-4e7c-b729-6dea600ce8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.457918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:36.482269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.485421 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.535631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1231/2478] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:54:36.540219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d42bc-cb3e-4e7c-b729-6dea600ce8e1 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.547951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:36.568448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b6e14-245c-4356-9cd8-316926deb000 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.574403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.575267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.575550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.575840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] PATH_INFO: `/v3/projects/89cd37f7227940b59dbcceb9525c9704` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.577107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89cd37f7227940b59dbcceb9525c9704)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.579019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.603997 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d42bc-cb3e-4e7c-b729-6dea600ce8e1 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.610962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.611675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.611959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.612232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.612962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.614848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.640901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89cd37f7227940b59dbcceb9525c9704', 'request_id': 'req-84f9c319-154b-48d7-9240-117dab1a4e65'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.646220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1232/2479] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:54:36 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:36.650293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.651214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89cd37f7227940b59dbcceb9525c9704', 'request_id': 'req-84f9c319-154b-48d7-9240-117dab1a4e65'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.659076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89cd37f7227940b59dbcceb9525c9704', 'request_id': 'req-84f9c319-154b-48d7-9240-117dab1a4e65'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.674160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] The token cache is being invalidate because project 89cd37f7227940b59dbcceb9525c9704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:36.674554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.676103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-84f9c319-154b-48d7-9240-117dab1a4e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:36.681814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1248/2480] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:36 2022] DELETE /identity/v3/projects/89cd37f7227940b59dbcceb9525c9704 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:36.683648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d696b-53cc-48aa-bc56-142d278ed7a0 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.683947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d696b-53cc-48aa-bc56-142d278ed7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.684231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d696b-53cc-48aa-bc56-142d278ed7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.725734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8b9d696b-53cc-48aa-bc56-142d278ed7a0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:36.727516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8b9d696b-53cc-48aa-bc56-142d278ed7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.728193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc3f22-1904-426a-89ff-5113046e07fd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.736265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.737470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.737751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.738024 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.738769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.740753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.753072 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1249/2481] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:36.754991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed04c5-69a7-40f3-9a26-b9cc0d6734dd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.755429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed04c5-69a7-40f3-9a26-b9cc0d6734dd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.756606 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed04c5-69a7-40f3-9a26-b9cc0d6734dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.784831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1233/2482] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:54:36 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:36.788548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.793453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.821358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-0fed04c5-69a7-40f3-9a26-b9cc0d6734dd None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:36.826565 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-0fed04c5-69a7-40f3-9a26-b9cc0d6734dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.829614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20552931-9bcb-4ff9-bcd5-c004edad2791 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.837438 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.838464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.839527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.840109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.841388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.841781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1250/2483] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:36.844378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.846257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed04c5-69a7-40f3-9a26-b9cc0d6734dd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.862819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1234/2484] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:36.866566 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.874576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed04c5-69a7-40f3-9a26-b9cc0d6734dd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.882531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934de05-bcc3-49fc-891a-6f34fa2ce4a7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.887807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.888328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.888666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.888989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.890396 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:54:36.890941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.892522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.905638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.906518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.907242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.907371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.908132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.909166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1251/2485] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:54:36.910668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.914239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.914408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:36.930733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1235/2486] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:36.931616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:36.937404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:36.943473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861852f9-f3ce-4f7a-a205-f5b3f7ca0971 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:36.950173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:36.951217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:36.951727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:36.952685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:36.953486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:36.955447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:36.990656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1236/2487] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:54:36 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:54:36.991590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.037852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badba39b-092c-4551-925d-fcc85e5ed2f6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.066267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412a3a8-8754-4a3d-a25c-179d9c0f1ced admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.067234 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:37.074168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.075787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.078101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.078484 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] PATH_INFO: `/v3/projects/5595f20f622147dd9526b7c1d78b0891/users/fbeb413d4e0a473882ec4ec3d995ae7c/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.097271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5595f20f622147dd9526b7c1d78b0891, user_id=fbeb413d4e0a473882ec4ec3d995ae7c, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.100297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.112008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a2b1-1190-4e55-9002-b5c5467d4581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.112812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a2b1-1190-4e55-9002-b5c5467d4581 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.113352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a2b1-1190-4e55-9002-b5c5467d4581 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.113487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a2b1-1190-4e55-9002-b5c5467d4581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.115800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-add2a2b1-1190-4e55-9002-b5c5467d4581 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:37.122927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1237/2488] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:36 2022] PUT /identity/v3/projects/5595f20f622147dd9526b7c1d78b0891/users/fbeb413d4e0a473882ec4ec3d995ae7c/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:37.125491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.131932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a2b1-1190-4e55-9002-b5c5467d4581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.133724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a2b1-1190-4e55-9002-b5c5467d4581 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.156615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1252/2489] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:37.158097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df937922-9124-4fb1-bbb1-7ab10241197b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.158389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df937922-9124-4fb1-bbb1-7ab10241197b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.158672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df937922-9124-4fb1-bbb1-7ab10241197b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.223507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f6760-19ca-4cba-bd14-173613950d36 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.231936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-df937922-9124-4fb1-bbb1-7ab10241197b None None] MFA Rules not processed for user `fbeb413d4e0a473882ec4ec3d995ae7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:37.234089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.235018 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.243453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.243453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.243453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.243453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.243947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-df937922-9124-4fb1-bbb1-7ab10241197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:37.290737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1238/2490] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:54:37.295528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.319644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8af857-4fb3-4dc8-8936-fe569ffd2f25 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.335870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.336616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.336890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.337362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.337983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.339917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.348603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1253/2491] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:37.350790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df937922-9124-4fb1-bbb1-7ab10241197b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.362279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1239/2492] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:37.365273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.365894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df937922-9124-4fb1-bbb1-7ab10241197b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.371265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.372062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.372529 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.373202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.376519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.377081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.381765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124caa0-a0a6-4475-b7df-797d7f912576 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.388497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.390112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.390536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.391560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.392312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.394657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.414542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1240/2493] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:54:37.419233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.419800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1254/2494] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:37.424000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.430662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59085ec0-722f-438f-9f30-ecf2cafb9428 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.466272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:37.499450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f8909-9c2d-48ab-97dc-0257be795715 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.499712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50cdc6-7a05-4e09-8f19-ac5d7f54109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.500693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50cdc6-7a05-4e09-8f19-ac5d7f54109a admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.500978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50cdc6-7a05-4e09-8f19-ac5d7f54109a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.501251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50cdc6-7a05-4e09-8f19-ac5d7f54109a admin admin] PATH_INFO: `/v3/projects/b2877ec8c29249ca83d2e75dad7d13be/users/3900f2a435af4626b1b9c39dc9c547b7/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.504258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.504983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.505265 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.505547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.506224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.508142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.517670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50cdc6-7a05-4e09-8f19-ac5d7f54109a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2877ec8c29249ca83d2e75dad7d13be, user_id=3900f2a435af4626b1b9c39dc9c547b7, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.521035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50cdc6-7a05-4e09-8f19-ac5d7f54109a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.530138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1255/2495] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:37.533985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.539832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1241/2496] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:37 2022] PUT /identity/v3/projects/b2877ec8c29249ca83d2e75dad7d13be/users/3900f2a435af4626b1b9c39dc9c547b7/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:37.544006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccda495-4a40-4654-8539-3a094b871f2a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.551626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef187030-9222-49bf-b727-94c451e3f65d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.551988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef187030-9222-49bf-b727-94c451e3f65d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.552339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef187030-9222-49bf-b727-94c451e3f65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.556104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:37.580047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ef187030-9222-49bf-b727-94c451e3f65d None None] MFA Rules not processed for user `3900f2a435af4626b1b9c39dc9c547b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:37.581331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ef187030-9222-49bf-b727-94c451e3f65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:37.607393 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.608641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.608927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.609202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.609933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.611537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.621599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1256/2497] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:54:37.626279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.636061 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586263e-ea2a-4682-a3bd-a6a160d0bb14 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.640961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.641784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.642067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.642344 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] PATH_INFO: `/v3/projects/36e9b45d41b94fe0ba5faa6e15f703b9/users/71663bcea87a4564998848ae44c23282/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.653356 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1242/2498] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:54:37.657911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef187030-9222-49bf-b727-94c451e3f65d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.662272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e9b45d41b94fe0ba5faa6e15f703b9, user_id=71663bcea87a4564998848ae44c23282, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.665009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.669151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef187030-9222-49bf-b727-94c451e3f65d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.679044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.679878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.680161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.680434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.681170 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.683067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.685286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1257/2499] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:37 2022] PUT /identity/v3/projects/36e9b45d41b94fe0ba5faa6e15f703b9/users/71663bcea87a4564998848ae44c23282/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:37.690339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.714998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1243/2500] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:37.720622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.763998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bcb0e-2913-4d32-88d3-d3afbb23e6f6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.784192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c2c7-dde7-404f-a64f-017dbd9b4f1c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.789048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.789719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.790006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.790285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.791011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.792966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.826385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1244/2501] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:54:37.827981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.829164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.829838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.830128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.830402 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.831130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.832787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.837972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf2fd-a7c2-4071-8917-0e7798c796bd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.842608 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.842970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1258/2502] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:54:37.843262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.843582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.843827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.844578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.846443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.846991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.851839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:37.857670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6b023-6ce8-4e4b-b341-aa96d1eb8864 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.859572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1245/2503] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:37.863802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.866028 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bc81f-f8ab-4ca5-9579-2adcab9137e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.866930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc81f-f8ab-4ca5-9579-2adcab9137e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.867854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc81f-f8ab-4ca5-9579-2adcab9137e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.868134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc81f-f8ab-4ca5-9579-2adcab9137e1 admin admin] PATH_INFO: `/v3/projects/36e9b45d41b94fe0ba5faa6e15f703b9/users/71663bcea87a4564998848ae44c23282/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.870281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bc81f-f8ab-4ca5-9579-2adcab9137e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e9b45d41b94fe0ba5faa6e15f703b9, user_id=71663bcea87a4564998848ae44c23282, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.872988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bc81f-f8ab-4ca5-9579-2adcab9137e1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.874512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904963a4-076d-4f98-99e4-e76679930b77 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:37.879374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:37.880301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.880596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.880887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] PATH_INFO: `/v3/projects/1f759e9436d4497f8ef6a8ecc4f604b2/users/ffe9120727ad446e90023ce2d7eb231c/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.888083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1259/2504] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:37 2022] PUT /identity/v3/projects/36e9b45d41b94fe0ba5faa6e15f703b9/users/71663bcea87a4564998848ae44c23282/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:37.899304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:37.899605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:37.899871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:37.900409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f759e9436d4497f8ef6a8ecc4f604b2, user_id=ffe9120727ad446e90023ce2d7eb231c, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:37.903474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:37.920035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1246/2505] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:37 2022] PUT /identity/v3/projects/1f759e9436d4497f8ef6a8ecc4f604b2/users/ffe9120727ad446e90023ce2d7eb231c/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:37.925301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:37.929739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:37.933023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] MFA Rules not processed for user `71663bcea87a4564998848ae44c23282`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:37.935456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:38.017186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aad3e7-e7e2-41e3-8174-a60b177b2383 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:38.022436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:38.023468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:38.023820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:38.024197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:38.025053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:38.026924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:38.041357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1247/2506] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:38.042125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1260/2507] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:54:38.043250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:38.055569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:38.060018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:38.063879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7beb59-536e-4fed-b7ad-918124f69e7c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:38.069522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade8980-3b7a-41e0-a6bb-6d8e0c7522c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:38.070441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade8980-3b7a-41e0-a6bb-6d8e0c7522c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:38.070724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade8980-3b7a-41e0-a6bb-6d8e0c7522c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:38.071000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade8980-3b7a-41e0-a6bb-6d8e0c7522c6 admin admin] PATH_INFO: `/v3/projects/1f759e9436d4497f8ef6a8ecc4f604b2/users/ffe9120727ad446e90023ce2d7eb231c/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:38.073579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade8980-3b7a-41e0-a6bb-6d8e0c7522c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f759e9436d4497f8ef6a8ecc4f604b2, user_id=ffe9120727ad446e90023ce2d7eb231c, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:38.075731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade8980-3b7a-41e0-a6bb-6d8e0c7522c6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:38.086883 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1248/2508] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:38 2022] PUT /identity/v3/projects/1f759e9436d4497f8ef6a8ecc4f604b2/users/ffe9120727ad446e90023ce2d7eb231c/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:38.097062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:38.097350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:38.097633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:38.125431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] MFA Rules not processed for user `ffe9120727ad446e90023ce2d7eb231c`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:38.126646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:38.132461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2ce8e-cddd-4614-ba46-a7a15a1b51be None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:38.185792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:38.186898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:38.187189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:38.187467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:38.189357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:38.202648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1249/2509] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:54:38.213891 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:38.215306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:38.227504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dab966-9803-4ee8-ad59-3e581cd680d3 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:38.231347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:38.231990 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:38.232278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:38.232562 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:38.242778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:38.244218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:38.254822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1250/2510] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:54:38.258799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:38.260447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:38.270025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1261/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:54:38.461616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:38.474373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d3117-419e-48a1-be16-fac20af4e03f service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:38.478379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:38.479184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:38.479529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:38.479888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:38.516467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:38.548182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:38.549120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:38.567969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1251/2512] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:40.024989 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:40.035720 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb034-7a90-47eb-a41f-8dc2ade3de39 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:40.039708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:40.040605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:40.040811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:40.040977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:40.046764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:40.114039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:40.115916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:40.128316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1262/2513] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:40.695407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:40.703996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65d18-a110-4267-bef5-415d23397518 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:40.707666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d98d45-6d49-4378-b460-b2706aa185d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:40.708299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98d45-6d49-4378-b460-b2706aa185d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:40.708615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98d45-6d49-4378-b460-b2706aa185d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:40.708900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98d45-6d49-4378-b460-b2706aa185d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:40.776645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98d45-6d49-4378-b460-b2706aa185d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:40.779162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98d45-6d49-4378-b460-b2706aa185d7 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:40.792022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1252/2514] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:49.110816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:49.183585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4c447-661c-4a14-b0e3-7e9503e51e00 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:49.188218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:49.189159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:49.189462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:49.189736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] PATH_INFO: `/v3/users/178a42a6e05240bd9343df3b3e380eaa` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:49.191231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] RBAC: Authorizing `identity:delete_user(user_id=178a42a6e05240bd9343df3b3e380eaa)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:49.194684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:49.247348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178a42a6e05240bd9343df3b3e380eaa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:49.256734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178a42a6e05240bd9343df3b3e380eaa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:49.264630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178a42a6e05240bd9343df3b3e380eaa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:49.265290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178a42a6e05240bd9343df3b3e380eaa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:49.265904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178a42a6e05240bd9343df3b3e380eaa'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:49.285791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1263/2515] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:49 2022] DELETE /identity/v3/users/178a42a6e05240bd9343df3b3e380eaa => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:49.290416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:49.291464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:49.291464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:49.308821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:49.310048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:49.382841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1253/2516] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:49.393413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:49.395004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:49.429096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:49.463135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46068e-d4eb-4a83-86a5-39eaaae4855b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:49.466839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:49.467591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:49.467872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:49.468136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:49.480016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:49.481337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:49.492581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1264/2517] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:49.964831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:49.966426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:49.993190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085b43b-ccb3-4cdc-9536-87130188dc8d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:49.998693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:49.999212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:49.999502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:49.999783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] PATH_INFO: `/v3/projects/9cdd89402d044366ad745bccaa0109bf` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:50.002366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cdd89402d044366ad745bccaa0109bf)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:50.005180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:50.061418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cdd89402d044366ad745bccaa0109bf', 'request_id': 'req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.072040 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cdd89402d044366ad745bccaa0109bf', 'request_id': 'req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.075247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:50.080929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cdd89402d044366ad745bccaa0109bf', 'request_id': 'req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.094908 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] The token cache is being invalidate because project 9cdd89402d044366ad745bccaa0109bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:50.095347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.096045 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.097638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1254/2518] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:49 2022] DELETE /identity/v3/projects/9cdd89402d044366ad745bccaa0109bf => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:50.101824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:50.116417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a72d-fdcb-437a-9da9-4b92a45580e6 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:50.178251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:50.179034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:50.179317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:50.179591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] PATH_INFO: `/v3/users/fbeb413d4e0a473882ec4ec3d995ae7c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:50.180876 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbeb413d4e0a473882ec4ec3d995ae7c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:50.182848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:50.237507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbeb413d4e0a473882ec4ec3d995ae7c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.245756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbeb413d4e0a473882ec4ec3d995ae7c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.253966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbeb413d4e0a473882ec4ec3d995ae7c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.254665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbeb413d4e0a473882ec4ec3d995ae7c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.255282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbeb413d4e0a473882ec4ec3d995ae7c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.274790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1265/2519] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:50 2022] DELETE /identity/v3/users/fbeb413d4e0a473882ec4ec3d995ae7c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:50.577519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:50.579346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:50.593842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ad4bc-b0b9-41e7-abf6-e82e187d1045 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:50.659953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:50.660861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:50.661146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:50.661423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] PATH_INFO: `/v3/projects/5595f20f622147dd9526b7c1d78b0891` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:50.662654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5595f20f622147dd9526b7c1d78b0891)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:50.665833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:50.716892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5595f20f622147dd9526b7c1d78b0891', 'request_id': 'req-3ef59408-9314-44ce-86ba-d3b6e89c33c1'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.728921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5595f20f622147dd9526b7c1d78b0891', 'request_id': 'req-3ef59408-9314-44ce-86ba-d3b6e89c33c1'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.736124 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5595f20f622147dd9526b7c1d78b0891', 'request_id': 'req-3ef59408-9314-44ce-86ba-d3b6e89c33c1'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.750467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] The token cache is being invalidate because project 5595f20f622147dd9526b7c1d78b0891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:50.750758 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.751425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.753150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1255/2520] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:50 2022] DELETE /identity/v3/projects/5595f20f622147dd9526b7c1d78b0891 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:50.757180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:50.758715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:50.773436 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e934ec-2653-4b80-a38b-1a9e21ce9e5f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:50.823675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:50.844697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:50.845579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:50.845859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:50.846129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] PATH_INFO: `/v3/users/71663bcea87a4564998848ae44c23282` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:50.847316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] RBAC: Authorizing `identity:delete_user(user_id=71663bcea87a4564998848ae44c23282)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:50.849295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:50.904197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71663bcea87a4564998848ae44c23282'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.913145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71663bcea87a4564998848ae44c23282'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.922530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71663bcea87a4564998848ae44c23282'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.923429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71663bcea87a4564998848ae44c23282'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.924147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3883d6a3-72b5-4aed-bcf9-3d1a85c6337e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71663bcea87a4564998848ae44c23282'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:50.947754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1266/2521] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:50 2022] DELETE /identity/v3/users/71663bcea87a4564998848ae44c23282 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:51.318126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:51.319810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:51.335392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef59408-9314-44ce-86ba-d3b6e89c33c1 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:51.406103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:51.406995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.407275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.407559 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] PATH_INFO: `/v3/projects/36e9b45d41b94fe0ba5faa6e15f703b9` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.408829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e9b45d41b94fe0ba5faa6e15f703b9)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:51.410745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:51.434491 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:51.465253 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e9b45d41b94fe0ba5faa6e15f703b9', 'request_id': 'req-133ae639-559f-41ef-aba2-83503b5d0a92'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:51.476592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e9b45d41b94fe0ba5faa6e15f703b9', 'request_id': 'req-133ae639-559f-41ef-aba2-83503b5d0a92'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:51.482444 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e9b45d41b94fe0ba5faa6e15f703b9', 'request_id': 'req-133ae639-559f-41ef-aba2-83503b5d0a92'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:51.495357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] The token cache is being invalidate because project 36e9b45d41b94fe0ba5faa6e15f703b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:51.495650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:51.496310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-133ae639-559f-41ef-aba2-83503b5d0a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:51.498049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1256/2522] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:51 2022] DELETE /identity/v3/projects/36e9b45d41b94fe0ba5faa6e15f703b9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:51.504348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4525e-508a-45cd-94ab-79a2d6ab4cab None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.505182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4525e-508a-45cd-94ab-79a2d6ab4cab None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.505353 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4525e-508a-45cd-94ab-79a2d6ab4cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.524914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-16a4525e-508a-45cd-94ab-79a2d6ab4cab None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:51.525930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-16a4525e-508a-45cd-94ab-79a2d6ab4cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:51.592707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1267/2523] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:51.601816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b734-adc6-4d36-9978-19ed7b858b8b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.602369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b734-adc6-4d36-9978-19ed7b858b8b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.602369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b734-adc6-4d36-9978-19ed7b858b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.624107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e182b734-adc6-4d36-9978-19ed7b858b8b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:51.625374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e182b734-adc6-4d36-9978-19ed7b858b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:51.636335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1257/2524] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:51.640952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4525e-508a-45cd-94ab-79a2d6ab4cab None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:51.651106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4525e-508a-45cd-94ab-79a2d6ab4cab None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:51.656964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:51.657918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.657918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.658549 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.662042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:54:51.662226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:51.663775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:51.680573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1268/2525] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:54:51.685135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182b734-adc6-4d36-9978-19ed7b858b8b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:51.695248 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182b734-adc6-4d36-9978-19ed7b858b8b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:51.700245 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:51.700938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.701224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.702014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.702740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:51.704707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:51.733190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1258/2526] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:54:51 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:54:51.737865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:51.752113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e077c5-762a-4cd3-9b82-1f9d5ee7c8b7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:51.808687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:51.817029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:51.819163 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.819756 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.820211 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.821915 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:51.823906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:51.849701 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1269/2527] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:54:51 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:54:51.856254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:51.869053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbdb1d-7c1d-494a-b45b-8fceb96f7531 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:51.875518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:51.876217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.876480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.876790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.877569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:51.879277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:51.890702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1259/2528] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:54:51.894815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:51.905199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c85a2-59a0-42e7-a970-65c55f433143 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:51.909867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87619c19-8746-428e-9520-2ca79e820042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:51.910521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87619c19-8746-428e-9520-2ca79e820042 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.910805 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87619c19-8746-428e-9520-2ca79e820042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.911082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87619c19-8746-428e-9520-2ca79e820042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.911840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87619c19-8746-428e-9520-2ca79e820042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:51.913462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87619c19-8746-428e-9520-2ca79e820042 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:51.925350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1270/2529] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:54:51.930143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:51.941919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5620e25-712a-483d-af83-6edafa3a3239 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:51.946995 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:51.947969 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.948335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.948719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] PATH_INFO: `/v3/projects/fe625d7fd44a439683fc1d2bc4cd6cb5/users/2e0e6c1ad20a48a38b9e9c5b446334fb/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:51.965738 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe625d7fd44a439683fc1d2bc4cd6cb5, user_id=2e0e6c1ad20a48a38b9e9c5b446334fb, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:51.968502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:51.982333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1260/2530] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:54:51 2022] PUT /identity/v3/projects/fe625d7fd44a439683fc1d2bc4cd6cb5/users/2e0e6c1ad20a48a38b9e9c5b446334fb/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:51.994715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05693954-dec8-429e-9dec-2ff184094277 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:51.995006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05693954-dec8-429e-9dec-2ff184094277 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:51.995291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05693954-dec8-429e-9dec-2ff184094277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:52.026250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-05693954-dec8-429e-9dec-2ff184094277 None None] MFA Rules not processed for user `2e0e6c1ad20a48a38b9e9c5b446334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:52.027472 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-05693954-dec8-429e-9dec-2ff184094277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:52.102699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1271/2531] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:54:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:54:52.123809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:52.126065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:52.231554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2f04-bd43-4118-b966-11194d086584 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:52.237150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28a07f47b8954418a5a86f344fb7fbe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:52.238206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:52.238694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:52.239039 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:52.248742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:52.250178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:52.259539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1261/2532] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:02.097799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:02.255183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:08.018967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05693954-dec8-429e-9dec-2ff184094277 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:08.020375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-05693954-dec8-429e-9dec-2ff184094277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:08.103304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05693954-dec8-429e-9dec-2ff184094277 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:08.109411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3117da8-1578-4a2b-a7a8-68251d39a9cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:08.110151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3117da8-1578-4a2b-a7a8-68251d39a9cf service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:08.110488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3117da8-1578-4a2b-a7a8-68251d39a9cf service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:08.110780 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3117da8-1578-4a2b-a7a8-68251d39a9cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:08.112801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c3117da8-1578-4a2b-a7a8-68251d39a9cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:08.192446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3117da8-1578-4a2b-a7a8-68251d39a9cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:08.194077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3117da8-1578-4a2b-a7a8-68251d39a9cf service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:08.204277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1272/2533] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:09.091662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:09.093882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:09.197476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21666dd6-8e2a-4478-a0b9-e9317c40624c service glance] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:09.211867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:09.211867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:09.211867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:09.211867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] PATH_INFO: `/v3/users/f67af7f314804e4d9d7cdccfb36f26e7` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:09.212860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67af7f314804e4d9d7cdccfb36f26e7)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:09.212860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:09.295237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67af7f314804e4d9d7cdccfb36f26e7'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:09.305260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67af7f314804e4d9d7cdccfb36f26e7'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:09.316388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67af7f314804e4d9d7cdccfb36f26e7'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:09.317508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67af7f314804e4d9d7cdccfb36f26e7'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:09.321103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67af7f314804e4d9d7cdccfb36f26e7'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:09.354022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1262/2534] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:09 2022] DELETE /identity/v3/users/f67af7f314804e4d9d7cdccfb36f26e7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:09.358739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:09.364504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:09.364504 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:09.380240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:09.381692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:09.470628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1273/2535] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:55:09.480453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:09.481945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:09.561032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f76185-d0d7-4d89-8c41-20bb70657efc admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:09.565658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a665a-5754-4820-a6af-c9e5641fba9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:09.566460 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a665a-5754-4820-a6af-c9e5641fba9c service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:09.566776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a665a-5754-4820-a6af-c9e5641fba9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:09.567201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a665a-5754-4820-a6af-c9e5641fba9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:09.581560 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a665a-5754-4820-a6af-c9e5641fba9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:09.583572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a665a-5754-4820-a6af-c9e5641fba9c service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:09.595037 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1263/2536] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:09.907199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:09.911287 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:09.947863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bc468-f28d-4092-93ef-a5da94a2e072 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:09.957466 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:09.958365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:09.958651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:09.958926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] PATH_INFO: `/v3/projects/1d08c10f0feb4953a01ae37166636ed3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:09.960213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d08c10f0feb4953a01ae37166636ed3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:09.962329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:09.994486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=100 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:10.022887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d08c10f0feb4953a01ae37166636ed3', 'request_id': 'req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:10.032302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d08c10f0feb4953a01ae37166636ed3', 'request_id': 'req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:10.043079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d08c10f0feb4953a01ae37166636ed3', 'request_id': 'req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:10.057271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] The token cache is being invalidate because project 1d08c10f0feb4953a01ae37166636ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:10.057671 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:10.058310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-622cd1e3-cfdc-4762-a9f6-fdba76cd57e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:10.059883 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1274/2537] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:09 2022] DELETE /identity/v3/projects/1d08c10f0feb4953a01ae37166636ed3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:10.065044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235543-7d24-4355-8a02-2587a5ad5a51 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.065918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235543-7d24-4355-8a02-2587a5ad5a51 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.066465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235543-7d24-4355-8a02-2587a5ad5a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.093091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-93235543-7d24-4355-8a02-2587a5ad5a51 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:10.094598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-93235543-7d24-4355-8a02-2587a5ad5a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:10.114534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:10.185432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1264/2538] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:55:10.193365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033cab-969c-4672-b9e9-7c67aaae72ed None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.193797 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033cab-969c-4672-b9e9-7c67aaae72ed None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.194148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033cab-969c-4672-b9e9-7c67aaae72ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.214347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-c4033cab-969c-4672-b9e9-7c67aaae72ed None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:10.215657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c4033cab-969c-4672-b9e9-7c67aaae72ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:10.225519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1275/2539] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:10.229915 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235543-7d24-4355-8a02-2587a5ad5a51 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.239759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235543-7d24-4355-8a02-2587a5ad5a51 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:10.244403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:10.245103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.245388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.245666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.246386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:55:10.246693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:10.248287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:10.268455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1265/2540] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:55:10.269601 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033cab-969c-4672-b9e9-7c67aaae72ed None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.279499 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033cab-969c-4672-b9e9-7c67aaae72ed None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:10.285022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:10.285689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.285974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.286253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.286955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:10.288919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:10.327386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1276/2541] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:55:10.329139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.340934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac90c3-d90d-4784-98ee-6d8d428b8e4d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:10.408128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:10.408867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.409150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.409429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.410129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:10.412152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:10.438652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1266/2542] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:10.443450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.464023 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202ed2e-ad28-457d-b987-bb4dc4460c14 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:10.475235 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84177bb6-b373-4391-a874-7538dc1d37e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:10.475700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84177bb6-b373-4391-a874-7538dc1d37e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.476401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84177bb6-b373-4391-a874-7538dc1d37e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.476858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84177bb6-b373-4391-a874-7538dc1d37e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.477685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84177bb6-b373-4391-a874-7538dc1d37e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:10.480483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84177bb6-b373-4391-a874-7538dc1d37e7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:10.497017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1277/2543] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:10.498589 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.510946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571428af-823b-40fe-9e43-e2fd85796665 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:10.516753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:10.517455 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.517755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.518013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.518739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:10.520531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:10.531587 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff227e9-ca74-4b74-ac21-f2241cfd595a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.531880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff227e9-ca74-4b74-ac21-f2241cfd595a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.532190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff227e9-ca74-4b74-ac21-f2241cfd595a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.532327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1267/2544] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:10.536984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.556917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-dff227e9-ca74-4b74-ac21-f2241cfd595a None None] MFA Rules not processed for user `f8333f6a9dab4618bc0e6b2060f15250`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:10.558155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dff227e9-ca74-4b74-ac21-f2241cfd595a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:10.558289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7df534-5513-4ce9-8b66-a1db6fb53482 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:10.563300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad17b9-84f7-4beb-b38f-ede80b6cbebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:10.565280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad17b9-84f7-4beb-b38f-ede80b6cbebf admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.565623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad17b9-84f7-4beb-b38f-ede80b6cbebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.565905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad17b9-84f7-4beb-b38f-ede80b6cbebf admin admin] PATH_INFO: `/v3/projects/ee42da8284cb429b8e834938ab98e604/users/81347117a1a2456d80cc845a991cfecc/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.584717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad17b9-84f7-4beb-b38f-ede80b6cbebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee42da8284cb429b8e834938ab98e604, user_id=81347117a1a2456d80cc845a991cfecc, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:10.587926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad17b9-84f7-4beb-b38f-ede80b6cbebf admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:10.611396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1268/2545] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:10 2022] PUT /identity/v3/projects/ee42da8284cb429b8e834938ab98e604/users/81347117a1a2456d80cc845a991cfecc/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:10.624213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6236db-ba48-430d-bb28-c573806227ef None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.624789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6236db-ba48-430d-bb28-c573806227ef None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.625650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6236db-ba48-430d-bb28-c573806227ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.648015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1278/2546] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:10.673089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-6c6236db-ba48-430d-bb28-c573806227ef None None] MFA Rules not processed for user `81347117a1a2456d80cc845a991cfecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:10.674887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-6c6236db-ba48-430d-bb28-c573806227ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:10.780571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:10.803569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff227e9-ca74-4b74-ac21-f2241cfd595a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.808914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1269/2547] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:55:10.819272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff227e9-ca74-4b74-ac21-f2241cfd595a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:10.883856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:10.884449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:10.884757 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:10.885039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:10.885814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:10.887718 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:10.917666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:10.927584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1279/2548] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:10.931744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6236db-ba48-430d-bb28-c573806227ef None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:10.943937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6236db-ba48-430d-bb28-c573806227ef None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:11.029926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:11.030601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:11.030905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:11.031178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:11.031865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:11.034604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:11.056333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1270/2549] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:10 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:11.060486 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:11.074282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d91fa4-ca6a-4e9d-b86f-470fcff45e0e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:11.079996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:11.080799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:11.081156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:11.081501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:11.082306 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:11.083975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:11.099616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1280/2550] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:11.100888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:11.112919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf02b27-7b34-48e1-884c-5b5c2c28f473 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:11.117811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:11.118710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:11.118998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:11.119276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] PATH_INFO: `/v3/projects/ad75b4f1395d431aa8e215f0d9dadc9c/users/f0dd8fd68d6c4171baa4668fbcf10425/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:11.136023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad75b4f1395d431aa8e215f0d9dadc9c, user_id=f0dd8fd68d6c4171baa4668fbcf10425, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:11.138638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:11.153776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1271/2551] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:11 2022] PUT /identity/v3/projects/ad75b4f1395d431aa8e215f0d9dadc9c/users/f0dd8fd68d6c4171baa4668fbcf10425/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:11.157785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:11.169455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a326343-a62f-4bdb-8fa2-4c5c5dd2d71b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:11.240588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be044e1-2689-4cb3-a0b5-02f289a46a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:11.241358 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be044e1-2689-4cb3-a0b5-02f289a46a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:11.241661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be044e1-2689-4cb3-a0b5-02f289a46a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:11.241948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be044e1-2689-4cb3-a0b5-02f289a46a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:11.242788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be044e1-2689-4cb3-a0b5-02f289a46a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:11.244430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be044e1-2689-4cb3-a0b5-02f289a46a61 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:11.260164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1281/2552] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:11.264276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:11.276381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d22206-ac57-4d7e-b29a-66c72c74eb11 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:11.282084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:11.283066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:11.283355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:11.283629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] PATH_INFO: `/v3/projects/ad75b4f1395d431aa8e215f0d9dadc9c/users/f0dd8fd68d6c4171baa4668fbcf10425/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:11.288196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad75b4f1395d431aa8e215f0d9dadc9c, user_id=f0dd8fd68d6c4171baa4668fbcf10425, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:11.291131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:11.305264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1272/2553] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:11 2022] PUT /identity/v3/projects/ad75b4f1395d431aa8e215f0d9dadc9c/users/f0dd8fd68d6c4171baa4668fbcf10425/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:11.318156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0385-bc57-4076-a2b1-4daf43c239c9 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:11.318455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0385-bc57-4076-a2b1-4daf43c239c9 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:11.318737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab0385-bc57-4076-a2b1-4daf43c239c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:11.355310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cbab0385-bc57-4076-a2b1-4daf43c239c9 None None] MFA Rules not processed for user `f0dd8fd68d6c4171baa4668fbcf10425`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:11.356550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cbab0385-bc57-4076-a2b1-4daf43c239c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:11.438021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1282/2554] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:11.450743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:11.452642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:11.517556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c93208-1c8a-47db-b69c-d9d5c5865974 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:11.521533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f76ec9-2a88-41e8-b3d1-d8b584e499fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:11.522314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76ec9-2a88-41e8-b3d1-d8b584e499fe service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:11.522708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76ec9-2a88-41e8-b3d1-d8b584e499fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:11.523120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76ec9-2a88-41e8-b3d1-d8b584e499fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:11.533820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f76ec9-2a88-41e8-b3d1-d8b584e499fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:11.535280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f76ec9-2a88-41e8-b3d1-d8b584e499fe service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:11.546249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1273/2555] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:11 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:12.047411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab0385-bc57-4076-a2b1-4daf43c239c9 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:12.162320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab0385-bc57-4076-a2b1-4daf43c239c9 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:12.167600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:12.169450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:12.169450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:12.169717 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] PATH_INFO: `/v3/users/81347117a1a2456d80cc845a991cfecc` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:12.170955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81347117a1a2456d80cc845a991cfecc)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:12.173746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:12.191264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:12.258592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81347117a1a2456d80cc845a991cfecc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:12.275416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81347117a1a2456d80cc845a991cfecc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:12.286534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81347117a1a2456d80cc845a991cfecc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:12.287298 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81347117a1a2456d80cc845a991cfecc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:12.288186 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81347117a1a2456d80cc845a991cfecc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:12.343658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1283/2556] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:12 2022] DELETE /identity/v3/users/81347117a1a2456d80cc845a991cfecc => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:12.348422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:12.349361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:12.349944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:12.388465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:12.390251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:12.445513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:12.466858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1274/2557] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:12.478179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:12.479637 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:12.565053 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0de5e1-068b-48ba-8103-5b99604bc058 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:12.568956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:12.569651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:12.569990 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:12.570348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:12.583519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:12.585557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:12.598495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1284/2558] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:12.920163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:12.921805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:12.939762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbacfd-9815-41be-96f1-cbde5b0bdd4a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:12.944601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:12.945494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:12.945802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:12.946060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] PATH_INFO: `/v3/projects/ee42da8284cb429b8e834938ab98e604` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:12.947475 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee42da8284cb429b8e834938ab98e604)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:12.949546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:12.999800 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee42da8284cb429b8e834938ab98e604', 'request_id': 'req-e6261cfb-f0e7-4792-8736-afcc7f850911'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.008147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee42da8284cb429b8e834938ab98e604', 'request_id': 'req-e6261cfb-f0e7-4792-8736-afcc7f850911'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.014480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee42da8284cb429b8e834938ab98e604', 'request_id': 'req-e6261cfb-f0e7-4792-8736-afcc7f850911'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.026542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] The token cache is being invalidate because project ee42da8284cb429b8e834938ab98e604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:13.026833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.027539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.029129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1275/2559] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:12 2022] DELETE /identity/v3/projects/ee42da8284cb429b8e834938ab98e604 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:13.033684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:13.043841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2f97c-4a26-4a4e-b0f9-3a33d4924135 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:13.113381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:13.114251 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:13.114540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:13.114813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] PATH_INFO: `/v3/users/f0dd8fd68d6c4171baa4668fbcf10425` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:13.116012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0dd8fd68d6c4171baa4668fbcf10425)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:13.117971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:13.131331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:13.170538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0dd8fd68d6c4171baa4668fbcf10425'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.177958 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0dd8fd68d6c4171baa4668fbcf10425'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.186129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0dd8fd68d6c4171baa4668fbcf10425'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.186784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0dd8fd68d6c4171baa4668fbcf10425'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.187394 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f261c8fe-027e-4608-b32f-211820f6976d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0dd8fd68d6c4171baa4668fbcf10425'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.210420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1285/2560] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:13 2022] DELETE /identity/v3/users/f0dd8fd68d6c4171baa4668fbcf10425 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:13.598805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:13.601092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:13.619712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6261cfb-f0e7-4792-8736-afcc7f850911 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:13.685582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:13.687128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:13.687580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:13.688050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] PATH_INFO: `/v3/projects/ad75b4f1395d431aa8e215f0d9dadc9c` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:13.690668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad75b4f1395d431aa8e215f0d9dadc9c)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:13.694061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:13.744259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad75b4f1395d431aa8e215f0d9dadc9c', 'request_id': 'req-c13f4e9f-2096-40d8-8a70-403e9282a60a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.752359 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad75b4f1395d431aa8e215f0d9dadc9c', 'request_id': 'req-c13f4e9f-2096-40d8-8a70-403e9282a60a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.758949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad75b4f1395d431aa8e215f0d9dadc9c', 'request_id': 'req-c13f4e9f-2096-40d8-8a70-403e9282a60a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.772391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] The token cache is being invalidate because project ad75b4f1395d431aa8e215f0d9dadc9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:13.772723 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.773737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-c13f4e9f-2096-40d8-8a70-403e9282a60a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:13.775794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1276/2561] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:13 2022] DELETE /identity/v3/projects/ad75b4f1395d431aa8e215f0d9dadc9c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:13.781617 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d8197-06c6-48fb-8439-8bedccb480ec None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:13.781906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d8197-06c6-48fb-8439-8bedccb480ec None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:13.782178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d8197-06c6-48fb-8439-8bedccb480ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:13.801267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fb5d8197-06c6-48fb-8439-8bedccb480ec None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:13.802473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fb5d8197-06c6-48fb-8439-8bedccb480ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:13.951847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1286/2562] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:13.960069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e417fe7-cfc4-4695-b61f-4d34ace15d5e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:13.960353 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e417fe7-cfc4-4695-b61f-4d34ace15d5e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:13.960664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e417fe7-cfc4-4695-b61f-4d34ace15d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:13.968469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:14.000120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-3e417fe7-cfc4-4695-b61f-4d34ace15d5e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:14.003083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3e417fe7-cfc4-4695-b61f-4d34ace15d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:14.016224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1277/2563] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:14.023481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d8197-06c6-48fb-8439-8bedccb480ec None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.035307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d8197-06c6-48fb-8439-8bedccb480ec None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.043874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.044967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.045257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.045709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.046507 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:55:14.046815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.048493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.063328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1287/2564] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:55:14.069870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e417fe7-cfc4-4695-b61f-4d34ace15d5e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.081203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e417fe7-cfc4-4695-b61f-4d34ace15d5e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.087513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.088419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.088848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.089110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.089918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.091998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.129017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1278/2565] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:14 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:14.133057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.149145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970f939-4219-454d-a58a-7dca05d067df admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.253299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.254377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.254670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.254933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.255843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.258662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.285523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1288/2566] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:14 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:55:14.289348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.305188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d226e1-e21b-410a-b57a-d9a186ccb850 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.312398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.313130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.313418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.313726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.314463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.316343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.336793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1279/2567] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:14.339969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.354829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c964e4d-5aa5-4578-a619-c323753ff1bc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.363794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f23dd-89e3-41f0-97bc-e18933cf1e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.364933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f23dd-89e3-41f0-97bc-e18933cf1e19 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.364933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f23dd-89e3-41f0-97bc-e18933cf1e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.365318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f23dd-89e3-41f0-97bc-e18933cf1e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.366136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f23dd-89e3-41f0-97bc-e18933cf1e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.368068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f23dd-89e3-41f0-97bc-e18933cf1e19 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.377011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:14.386790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1289/2568] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:14.391921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.404111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d126a-2910-4bdd-a934-fe9f88635320 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.409866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.410761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.411044 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.411317 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] PATH_INFO: `/v3/projects/9d6cb3bb283a433fb2d6956eacd960d0/users/465d1764d53445f5b8a570f7de18d748/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.435618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6cb3bb283a433fb2d6956eacd960d0, user_id=465d1764d53445f5b8a570f7de18d748, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.438168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.460764 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1280/2569] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:14 2022] PUT /identity/v3/projects/9d6cb3bb283a433fb2d6956eacd960d0/users/465d1764d53445f5b8a570f7de18d748/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:14.467566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a1acd-b807-4e81-be82-15ac1dacb0d6 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.468043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a1acd-b807-4e81-be82-15ac1dacb0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.468511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a1acd-b807-4e81-be82-15ac1dacb0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.526685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ad4a1acd-b807-4e81-be82-15ac1dacb0d6 None None] MFA Rules not processed for user `465d1764d53445f5b8a570f7de18d748`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:14.527447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ad4a1acd-b807-4e81-be82-15ac1dacb0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:14.669613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1290/2570] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:14.675752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.774377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc37b44-0059-41ed-ade3-1839a6ccb9f9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.780422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.781440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.782173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.783197 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.784083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.786237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.821371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1281/2571] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:14 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:14.822650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a1acd-b807-4e81-be82-15ac1dacb0d6 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:14.834853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a1acd-b807-4e81-be82-15ac1dacb0d6 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:14.934664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:14.935984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:14.936320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:14.936628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:14.937798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:14.940521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:14.994580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1291/2572] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:14 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:14.998496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:15.027905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f50d5-480f-4e06-9aab-f136f5b3abe9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:15.044848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:15.047151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:15.047657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:15.049334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:15.051060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:15.054743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:15.084960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1282/2573] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:15.086125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:15.099888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d2280-60b1-4641-8c66-80f5512cbeea admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:15.112869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:15.114888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:15.115150 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:15.115432 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] PATH_INFO: `/v3/projects/891a161176794b37a0a0ba9a47d2228d/users/fad7c0cd8c924191983aee5913df3de4/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:15.169241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=891a161176794b37a0a0ba9a47d2228d, user_id=fad7c0cd8c924191983aee5913df3de4, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:15.171947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:15.204258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1292/2574] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:15 2022] PUT /identity/v3/projects/891a161176794b37a0a0ba9a47d2228d/users/fad7c0cd8c924191983aee5913df3de4/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:15.208978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:15.224334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645279d3-e9d8-468e-bde3-0f0c14ab6acf admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:15.341688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d74a22-6320-4eb5-89a3-5a6a4b97c2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:15.342443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74a22-6320-4eb5-89a3-5a6a4b97c2ab admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:15.342730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74a22-6320-4eb5-89a3-5a6a4b97c2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:15.342944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74a22-6320-4eb5-89a3-5a6a4b97c2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:15.343706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d74a22-6320-4eb5-89a3-5a6a4b97c2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:15.350952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d74a22-6320-4eb5-89a3-5a6a4b97c2ab admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:15.364737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1283/2575] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:15.368753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:15.381927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f7396-6148-46ba-adec-45813963b38f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:15.386994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:15.387902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:15.389090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:15.389090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] PATH_INFO: `/v3/projects/891a161176794b37a0a0ba9a47d2228d/users/fad7c0cd8c924191983aee5913df3de4/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:15.390545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891a161176794b37a0a0ba9a47d2228d, user_id=fad7c0cd8c924191983aee5913df3de4, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:15.393321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:15.413594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1293/2576] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:15 2022] PUT /identity/v3/projects/891a161176794b37a0a0ba9a47d2228d/users/fad7c0cd8c924191983aee5913df3de4/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:55:15.425563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2b651-2aca-4dc4-90c6-c9dad4f3964e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:15.425850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2b651-2aca-4dc4-90c6-c9dad4f3964e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:15.426310 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2b651-2aca-4dc4-90c6-c9dad4f3964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:15.450567 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:15.470308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ffe2b651-2aca-4dc4-90c6-c9dad4f3964e None None] MFA Rules not processed for user `fad7c0cd8c924191983aee5913df3de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:15.471606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ffe2b651-2aca-4dc4-90c6-c9dad4f3964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:15.549177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1284/2577] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:15.560708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:15.562407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:15.614245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:15.633210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827f8c1-57db-4fe2-a670-b69d9d4f2dc3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:15.642143 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:15.642889 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:15.643246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:15.643596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:15.654645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:15.656226 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:15.672090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1294/2578] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:15.758001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2b651-2aca-4dc4-90c6-c9dad4f3964e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:15.777775 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2b651-2aca-4dc4-90c6-c9dad4f3964e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:15.781394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33260e-1cc7-4b6f-99a0-f41af24e66ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:15.782043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33260e-1cc7-4b6f-99a0-f41af24e66ce service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:15.782321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33260e-1cc7-4b6f-99a0-f41af24e66ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:15.783098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33260e-1cc7-4b6f-99a0-f41af24e66ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:15.858254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33260e-1cc7-4b6f-99a0-f41af24e66ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:15.859699 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33260e-1cc7-4b6f-99a0-f41af24e66ce service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:15.869859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1285/2579] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:55:18.636468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:18.702957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0722b-259f-4c46-885e-a4975b9d4cab service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:18.707955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:18.708862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:18.709141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:18.709417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] PATH_INFO: `/v3/users/465d1764d53445f5b8a570f7de18d748` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:18.710681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=465d1764d53445f5b8a570f7de18d748)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:18.712708 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:18.760855 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465d1764d53445f5b8a570f7de18d748'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:18.768011 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465d1764d53445f5b8a570f7de18d748'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:18.775404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465d1764d53445f5b8a570f7de18d748'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:18.775962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465d1764d53445f5b8a570f7de18d748'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:18.779025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465d1764d53445f5b8a570f7de18d748'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:18.794350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1295/2580] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:18 2022] DELETE /identity/v3/users/465d1764d53445f5b8a570f7de18d748 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:18.799307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:18.799870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:18.800008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:18.817696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:18.818938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:18.883476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1286/2581] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:18.892930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:18.894337 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:18.952346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20149e4b-deb3-4f04-af3b-36df40e99a31 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:18.956017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:18.956702 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:18.956988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:18.957263 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:18.966189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:18.967610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:18.977645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1296/2582] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:19.247648 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:19.249308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:19.263202 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd55654-d924-4afc-9dd0-428d1c1a30a9 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:19.268435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:19.269293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:19.269703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:19.270463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] PATH_INFO: `/v3/projects/9d6cb3bb283a433fb2d6956eacd960d0` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:19.271235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6cb3bb283a433fb2d6956eacd960d0)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:19.273199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:19.307066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:19.322735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6cb3bb283a433fb2d6956eacd960d0', 'request_id': 'req-705cb201-0a77-4cde-a606-585a452fa414'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.330510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6cb3bb283a433fb2d6956eacd960d0', 'request_id': 'req-705cb201-0a77-4cde-a606-585a452fa414'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.336006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6cb3bb283a433fb2d6956eacd960d0', 'request_id': 'req-705cb201-0a77-4cde-a606-585a452fa414'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.347931 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] The token cache is being invalidate because project 9d6cb3bb283a433fb2d6956eacd960d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:19.348271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.348935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.350500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1287/2583] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:19 2022] DELETE /identity/v3/projects/9d6cb3bb283a433fb2d6956eacd960d0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:19.354462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:19.360749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:19.364948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74615d06-da88-4695-9c08-bec11839256f service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:19.424454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:19.425608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:19.425887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:19.426155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] PATH_INFO: `/v3/users/fad7c0cd8c924191983aee5913df3de4` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:19.427338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad7c0cd8c924191983aee5913df3de4)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:19.429284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:19.484667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad7c0cd8c924191983aee5913df3de4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.493837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad7c0cd8c924191983aee5913df3de4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.502535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad7c0cd8c924191983aee5913df3de4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.503209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad7c0cd8c924191983aee5913df3de4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.503831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-68ced6f4-c6ec-4be7-96de-f17948f67baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad7c0cd8c924191983aee5913df3de4'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.522766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1297/2584] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:19 2022] DELETE /identity/v3/users/fad7c0cd8c924191983aee5913df3de4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:19.798861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:19.800752 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:19.819075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cb201-0a77-4cde-a606-585a452fa414 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:19.881374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:19.882241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:19.882526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:19.882804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] PATH_INFO: `/v3/projects/891a161176794b37a0a0ba9a47d2228d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:19.883991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891a161176794b37a0a0ba9a47d2228d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:19.885979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:19.932030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891a161176794b37a0a0ba9a47d2228d', 'request_id': 'req-a1e307f7-5dcf-4e68-a399-c5fc300fe361'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.941176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891a161176794b37a0a0ba9a47d2228d', 'request_id': 'req-a1e307f7-5dcf-4e68-a399-c5fc300fe361'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.946784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891a161176794b37a0a0ba9a47d2228d', 'request_id': 'req-a1e307f7-5dcf-4e68-a399-c5fc300fe361'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.959130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] The token cache is being invalidate because project 891a161176794b37a0a0ba9a47d2228d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:19.959415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.960064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a1e307f7-5dcf-4e68-a399-c5fc300fe361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:19.961630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1288/2585] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:19 2022] DELETE /identity/v3/projects/891a161176794b37a0a0ba9a47d2228d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:19.966443 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc36c8d-0d5a-4483-a674-a56a92d9d630 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:19.966927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc36c8d-0d5a-4483-a674-a56a92d9d630 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:19.967033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc36c8d-0d5a-4483-a674-a56a92d9d630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:19.986935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7dc36c8d-0d5a-4483-a674-a56a92d9d630 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:19.988844 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7dc36c8d-0d5a-4483-a674-a56a92d9d630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:20.057168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1298/2586] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:20.065476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66b43-ceae-4271-8000-3d59b27f2edf None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.067181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66b43-ceae-4271-8000-3d59b27f2edf None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.069057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66b43-ceae-4271-8000-3d59b27f2edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.097837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-77e66b43-ceae-4271-8000-3d59b27f2edf None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:20.098974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-77e66b43-ceae-4271-8000-3d59b27f2edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:20.106926 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1289/2587] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:20.111239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc36c8d-0d5a-4483-a674-a56a92d9d630 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.121083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc36c8d-0d5a-4483-a674-a56a92d9d630 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.126888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.127545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.127825 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.128100 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.128856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:55:20.129259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.131944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.145284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1299/2588] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:55:20.149777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66b43-ceae-4271-8000-3d59b27f2edf None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.159150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66b43-ceae-4271-8000-3d59b27f2edf None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.163852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.164512 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.164824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.165102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.165799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.167683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.194059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1290/2589] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:20 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:20.198355 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.209246 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c81c-c87b-4d26-986a-74ae52c5b1a5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.217128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:20.268074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.268789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.269081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.269351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.270036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.271945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.291995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1300/2590] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:20 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:55:20.296118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.306870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003d57f-5b73-43a0-bf45-4f10ce8d25f6 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.311700 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.312494 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.312814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.313094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.314361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.315947 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.326912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1291/2591] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:20.331633 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.341597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab871a-0f6f-4a9d-9728-b114d85814f3 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.346189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178a2ed-103c-4cc6-809b-acfbd3e30264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.346845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178a2ed-103c-4cc6-809b-acfbd3e30264 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.347114 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178a2ed-103c-4cc6-809b-acfbd3e30264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.347390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178a2ed-103c-4cc6-809b-acfbd3e30264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.348207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178a2ed-103c-4cc6-809b-acfbd3e30264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.349745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178a2ed-103c-4cc6-809b-acfbd3e30264 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.359495 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1301/2592] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:20.363886 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.373856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769713c-a6d9-4790-b167-3b966b69c5be admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.379012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.379887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.380174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.380452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] PATH_INFO: `/v3/projects/70af532513c74085b0a7c3afd54ba4a9/users/a2e0b5107cc949e48668f53a0ab4d123/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.382694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:20.398285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70af532513c74085b0a7c3afd54ba4a9, user_id=a2e0b5107cc949e48668f53a0ab4d123, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.400959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.413702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1292/2593] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:20 2022] PUT /identity/v3/projects/70af532513c74085b0a7c3afd54ba4a9/users/a2e0b5107cc949e48668f53a0ab4d123/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:20.424863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3729031-14fa-4348-ab24-6041283a8810 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.425420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3729031-14fa-4348-ab24-6041283a8810 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.425698 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3729031-14fa-4348-ab24-6041283a8810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.451943 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f3729031-14fa-4348-ab24-6041283a8810 None None] MFA Rules not processed for user `a2e0b5107cc949e48668f53a0ab4d123`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:20.453167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f3729031-14fa-4348-ab24-6041283a8810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:20.515745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1302/2594] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:20.521036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.584014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31464ab-6bdd-4f59-b169-380f30502859 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.588754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.589415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.589698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.589975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.590675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.592583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.622400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1293/2595] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:20 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:55:20.623502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3729031-14fa-4348-ab24-6041283a8810 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.634777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3729031-14fa-4348-ab24-6041283a8810 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.693202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.694022 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.694314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.694740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.695435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.697423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.721863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1303/2596] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:20 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:55:20.723077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.733607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807acd4-ca2a-4e04-a7e5-3902ce381273 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.738192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.738841 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.739120 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.739394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.740139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.741762 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.752634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1294/2597] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:20.756666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.767399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2ea29-9581-472f-bbdd-a6e0c1e28dc6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.772170 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.773132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.773412 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.773684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] PATH_INFO: `/v3/projects/afe8ab451b024e7b91b281fbf596d91a/users/7306d699804d4003ab7e8e3aae2f285f/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.777581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:20.790572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe8ab451b024e7b91b281fbf596d91a, user_id=7306d699804d4003ab7e8e3aae2f285f, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.793302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.807728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1304/2598] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:20 2022] PUT /identity/v3/projects/afe8ab451b024e7b91b281fbf596d91a/users/7306d699804d4003ab7e8e3aae2f285f/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:20.811953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.821912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39f5a1-a2ff-4a4c-ab1d-d050a48f5a8d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.881098 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592341e-9208-4179-9d25-8e1179912d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.881781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592341e-9208-4179-9d25-8e1179912d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.882088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592341e-9208-4179-9d25-8e1179912d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.882369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592341e-9208-4179-9d25-8e1179912d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.883096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592341e-9208-4179-9d25-8e1179912d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.884733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592341e-9208-4179-9d25-8e1179912d27 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.898861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1295/2599] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:55:20.900037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:20.910376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1c59e-2815-49f4-ae15-f7eab7a69b70 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:20.914988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:20.915865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.916153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.916468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] PATH_INFO: `/v3/projects/afe8ab451b024e7b91b281fbf596d91a/users/7306d699804d4003ab7e8e3aae2f285f/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.918461 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe8ab451b024e7b91b281fbf596d91a, user_id=7306d699804d4003ab7e8e3aae2f285f, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:20.921173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:20.932517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1305/2600] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:20 2022] PUT /identity/v3/projects/afe8ab451b024e7b91b281fbf596d91a/users/7306d699804d4003ab7e8e3aae2f285f/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:20.943293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a12e5-fa0f-483c-ba01-ad59c1ca0480 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:20.943590 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a12e5-fa0f-483c-ba01-ad59c1ca0480 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:20.943874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a12e5-fa0f-483c-ba01-ad59c1ca0480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:20.973109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-027a12e5-fa0f-483c-ba01-ad59c1ca0480 None None] MFA Rules not processed for user `7306d699804d4003ab7e8e3aae2f285f`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:20.974366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-027a12e5-fa0f-483c-ba01-ad59c1ca0480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:21.022276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:21.054181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1296/2601] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:21.067164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:21.069059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:21.158440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a7756-b7ad-4500-8d9c-99bd1a5c5425 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:21.163464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f54786-2af1-4cd6-b802-1a0dc49c4da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:21.164439 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f54786-2af1-4cd6-b802-1a0dc49c4da0 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:21.164939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f54786-2af1-4cd6-b802-1a0dc49c4da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:21.165356 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f54786-2af1-4cd6-b802-1a0dc49c4da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:21.179676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f54786-2af1-4cd6-b802-1a0dc49c4da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:21.181200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f54786-2af1-4cd6-b802-1a0dc49c4da0 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:21.192610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1306/2602] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:21.276522 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a12e5-fa0f-483c-ba01-ad59c1ca0480 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:21.350861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a12e5-fa0f-483c-ba01-ad59c1ca0480 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:21.355932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:21.356872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:21.357161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:21.357436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] PATH_INFO: `/v3/users/a2e0b5107cc949e48668f53a0ab4d123` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:21.358641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e0b5107cc949e48668f53a0ab4d123)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:21.361132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:21.407020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e0b5107cc949e48668f53a0ab4d123'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:21.414190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e0b5107cc949e48668f53a0ab4d123'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:21.421307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e0b5107cc949e48668f53a0ab4d123'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:21.421948 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e0b5107cc949e48668f53a0ab4d123'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:21.422651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e0b5107cc949e48668f53a0ab4d123'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:21.441550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1297/2603] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:21 2022] DELETE /identity/v3/users/a2e0b5107cc949e48668f53a0ab4d123 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:21.446248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:21.446542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:21.446823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:21.465256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:21.466502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:21.526056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1307/2604] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:21.535659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:21.537182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:21.595622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428096-558d-4ada-aee3-81c22a2ad32d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:21.599271 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:21.599924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:21.600209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:21.600488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:21.610155 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:21.612094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:21.617111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:21.622257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1298/2605] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:21.922444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:21.924621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:21.937785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda22a3-b851-498f-8357-2cb9c314955f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:21.942523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:21.943384 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:21.943667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:21.943974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] PATH_INFO: `/v3/projects/70af532513c74085b0a7c3afd54ba4a9` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:21.945195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70af532513c74085b0a7c3afd54ba4a9)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:21.947704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:21.995903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70af532513c74085b0a7c3afd54ba4a9', 'request_id': 'req-7265f352-6be4-434d-878a-bef1e2d28042'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.005008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70af532513c74085b0a7c3afd54ba4a9', 'request_id': 'req-7265f352-6be4-434d-878a-bef1e2d28042'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.012247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70af532513c74085b0a7c3afd54ba4a9', 'request_id': 'req-7265f352-6be4-434d-878a-bef1e2d28042'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.020645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:22.025714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] The token cache is being invalidate because project 70af532513c74085b0a7c3afd54ba4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:22.026215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.027308 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.028966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1308/2606] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:21 2022] DELETE /identity/v3/projects/70af532513c74085b0a7c3afd54ba4a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:22.032917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:22.043672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd553-7dfb-445d-82d5-36a43c10d799 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:22.101604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:22.102474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:22.102774 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:22.103038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] PATH_INFO: `/v3/users/7306d699804d4003ab7e8e3aae2f285f` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:22.104209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7306d699804d4003ab7e8e3aae2f285f)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:22.106145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:22.151198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7306d699804d4003ab7e8e3aae2f285f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.157834 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7306d699804d4003ab7e8e3aae2f285f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.164941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7306d699804d4003ab7e8e3aae2f285f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.165613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7306d699804d4003ab7e8e3aae2f285f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.166218 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-726a3bda-f220-4de9-bb30-008d6751e4fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7306d699804d4003ab7e8e3aae2f285f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.184061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1299/2607] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:22 2022] DELETE /identity/v3/users/7306d699804d4003ab7e8e3aae2f285f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:22.493790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:22.495399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:22.509203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265f352-6be4-434d-878a-bef1e2d28042 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:22.574566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:22.575435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:22.575719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:22.575996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] PATH_INFO: `/v3/projects/afe8ab451b024e7b91b281fbf596d91a` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:22.577218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe8ab451b024e7b91b281fbf596d91a)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:22.579177 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:22.629949 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe8ab451b024e7b91b281fbf596d91a', 'request_id': 'req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.638290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe8ab451b024e7b91b281fbf596d91a', 'request_id': 'req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.644307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe8ab451b024e7b91b281fbf596d91a', 'request_id': 'req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.655626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] The token cache is being invalidate because project afe8ab451b024e7b91b281fbf596d91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:22.655909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.656638 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-1f5ac1c3-0f61-4a9f-8ae4-ac48966a2181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:22.658747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1309/2608] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:22 2022] DELETE /identity/v3/projects/afe8ab451b024e7b91b281fbf596d91a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:22.664395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee8717-adfc-4be8-b1da-ed5e3935b488 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:22.664395 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee8717-adfc-4be8-b1da-ed5e3935b488 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:22.664896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee8717-adfc-4be8-b1da-ed5e3935b488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:22.684842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-46ee8717-adfc-4be8-b1da-ed5e3935b488 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:22.686049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-46ee8717-adfc-4be8-b1da-ed5e3935b488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:22.756714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1300/2609] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:22.765248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f825f-6ad8-451c-a8a1-f4f67b3f52a3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:22.765570 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f825f-6ad8-451c-a8a1-f4f67b3f52a3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:22.765923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f825f-6ad8-451c-a8a1-f4f67b3f52a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:22.788643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-a85f825f-6ad8-451c-a8a1-f4f67b3f52a3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:22.789866 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a85f825f-6ad8-451c-a8a1-f4f67b3f52a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:22.798095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1310/2610] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:22.802779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee8717-adfc-4be8-b1da-ed5e3935b488 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:22.812835 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee8717-adfc-4be8-b1da-ed5e3935b488 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:22.817514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:22.818183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:22.819504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:22.819504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:22.819504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:55:22.819810 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:22.821367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:22.834503 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1301/2611] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:55:22.839902 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f825f-6ad8-451c-a8a1-f4f67b3f52a3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:22.850382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f825f-6ad8-451c-a8a1-f4f67b3f52a3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:22.855316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:22.855972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:22.856252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:22.856526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:22.857241 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:22.859128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:22.884383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1311/2612] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:22 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:22.888487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:22.899013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108e222-debc-4fd9-85e6-2c1696c55ba7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:22.904525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:22.964241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:22.965305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:22.965737 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:22.965866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:22.966655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:22.968684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:22.991174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1302/2613] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:22 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:22.995773 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.009745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b077-91b7-4731-ac25-2874f6aa6d85 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.014496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.015192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.015473 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.015749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.016489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.018679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.031051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1312/2614] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:23.036320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.047868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a254776-c090-4f3b-a586-ab74949438de admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.053082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf13c4-fa59-45e4-b156-e3aa32a8d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.053837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf13c4-fa59-45e4-b156-e3aa32a8d326 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.054215 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf13c4-fa59-45e4-b156-e3aa32a8d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.054561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf13c4-fa59-45e4-b156-e3aa32a8d326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.055357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf13c4-fa59-45e4-b156-e3aa32a8d326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.058408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf13c4-fa59-45e4-b156-e3aa32a8d326 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.076117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1303/2615] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:23.081247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.091326 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955dfda-5b9f-413c-b9fe-c979195c9678 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.096539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.097447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.097728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.098122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] PATH_INFO: `/v3/projects/3c96c719420c4597a5728951b75020e2/users/2022bc0df27249d18fe6e06414c00954/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.113068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c96c719420c4597a5728951b75020e2, user_id=2022bc0df27249d18fe6e06414c00954, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.115723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.129730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1313/2616] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:23 2022] PUT /identity/v3/projects/3c96c719420c4597a5728951b75020e2/users/2022bc0df27249d18fe6e06414c00954/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:55:23.194578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3ef63-f8ef-43ae-bf45-4ff673d330a8 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.194852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3ef63-f8ef-43ae-bf45-4ff673d330a8 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.195132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3ef63-f8ef-43ae-bf45-4ff673d330a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.225117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-abd3ef63-f8ef-43ae-bf45-4ff673d330a8 None None] MFA Rules not processed for user `2022bc0df27249d18fe6e06414c00954`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:23.226449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-abd3ef63-f8ef-43ae-bf45-4ff673d330a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:23.293288 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1304/2617] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:23.297966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.326794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:23.362920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22d119-02d3-47e3-b704-35b4cc7758af admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.367597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.368252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.368537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.368847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.369542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.371444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.399573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1314/2618] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:23 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:23.403959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3ef63-f8ef-43ae-bf45-4ff673d330a8 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.414600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3ef63-f8ef-43ae-bf45-4ff673d330a8 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.472261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.472957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.473240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.473515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.474200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.476085 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.495664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1305/2619] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:23 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:23.499832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.512108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f4438-a6ae-4f1e-b7e8-b9649a407527 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.518231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.518898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.519190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.519477 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.520210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.521874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.532847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1315/2620] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:23.537062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.543478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:23.547696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5592f5-954b-4bea-a87f-4c48ce6684d7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.554414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.555295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.555897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.555897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] PATH_INFO: `/v3/projects/6919ccaecee549da99603d9fc4008e30/users/448a5ec686534e908de8ef2696e2002e/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.572411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6919ccaecee549da99603d9fc4008e30, user_id=448a5ec686534e908de8ef2696e2002e, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.575107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.588422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1306/2621] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:23 2022] PUT /identity/v3/projects/6919ccaecee549da99603d9fc4008e30/users/448a5ec686534e908de8ef2696e2002e/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:23.592386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.603054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a49f0d-0a18-46fa-b5b3-a6d0c2abb593 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.669544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e0959-4fe1-4d84-b8ab-a99d9480b874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.670219 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e0959-4fe1-4d84-b8ab-a99d9480b874 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.670502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e0959-4fe1-4d84-b8ab-a99d9480b874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.670777 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e0959-4fe1-4d84-b8ab-a99d9480b874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.671501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e0959-4fe1-4d84-b8ab-a99d9480b874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.673158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e0959-4fe1-4d84-b8ab-a99d9480b874 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.683370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1316/2622] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:23.687493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.697695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec061b20-bdc7-4120-a47e-375f0cf0a2ae admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.702322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.703190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.703471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.703744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] PATH_INFO: `/v3/projects/6919ccaecee549da99603d9fc4008e30/users/448a5ec686534e908de8ef2696e2002e/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.706255 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6919ccaecee549da99603d9fc4008e30, user_id=448a5ec686534e908de8ef2696e2002e, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.708912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.719366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1307/2623] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:23 2022] PUT /identity/v3/projects/6919ccaecee549da99603d9fc4008e30/users/448a5ec686534e908de8ef2696e2002e/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:23.729885 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb63e5-e926-4648-b85b-0bf1dda4360b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.730174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb63e5-e926-4648-b85b-0bf1dda4360b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.730459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb63e5-e926-4648-b85b-0bf1dda4360b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.759595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-1fdb63e5-e926-4648-b85b-0bf1dda4360b None None] MFA Rules not processed for user `448a5ec686534e908de8ef2696e2002e`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:23.760831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1fdb63e5-e926-4648-b85b-0bf1dda4360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:23.828147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1317/2624] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:23.839108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:23.840885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:23.897537 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debcc76-7109-4ef9-986a-8f4c0257b30e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:23.901308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbf81b-c1d8-4d68-bdc1-990e6184ba00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:23.901814 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf81b-c1d8-4d68-bdc1-990e6184ba00 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:23.902092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf81b-c1d8-4d68-bdc1-990e6184ba00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:23.902366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf81b-c1d8-4d68-bdc1-990e6184ba00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:23.911192 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbf81b-c1d8-4d68-bdc1-990e6184ba00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:23.912629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbf81b-c1d8-4d68-bdc1-990e6184ba00 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:23.921783 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1308/2625] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:24.050610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb63e5-e926-4648-b85b-0bf1dda4360b None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:24.071343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:24.112372 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb63e5-e926-4648-b85b-0bf1dda4360b None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:24.116786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:24.117646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:24.118141 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:24.118282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] PATH_INFO: `/v3/users/2022bc0df27249d18fe6e06414c00954` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:24.119445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2022bc0df27249d18fe6e06414c00954)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:24.121483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:24.166974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2022bc0df27249d18fe6e06414c00954'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.174242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2022bc0df27249d18fe6e06414c00954'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.181540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2022bc0df27249d18fe6e06414c00954'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.182215 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2022bc0df27249d18fe6e06414c00954'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.182813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2022bc0df27249d18fe6e06414c00954'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.200535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1318/2626] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:24 2022] DELETE /identity/v3/users/2022bc0df27249d18fe6e06414c00954 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:24.205679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:24.206171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:24.206276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:24.224535 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:24.225845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:24.244441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:24.246149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:24.287895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1309/2627] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:24.297465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:24.299313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:24.344066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13fc57-d86b-4f70-8054-626e4dc6f270 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:24.349982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:24.350480 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:24.350759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:24.351032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:24.353193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:24.368152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dbf7e-b25c-4e2c-97aa-562d39c17c19 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:24.371694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:24.372351 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:24.372665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:24.372960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:24.381964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:24.383549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:24.385032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:24.395349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1310/2628] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:55:24.427679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:24.429367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:24.440314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1319/2629] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:24.856900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:24.858364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:24.882015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd2ee-b6b2-4b6f-b5f5-87be2b9fe8e8 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:24.898942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:24.899855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:24.900139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:24.900416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] PATH_INFO: `/v3/projects/3c96c719420c4597a5728951b75020e2` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:24.901697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c96c719420c4597a5728951b75020e2)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:24.903731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:24.912618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:24.957777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c96c719420c4597a5728951b75020e2', 'request_id': 'req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.966840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c96c719420c4597a5728951b75020e2', 'request_id': 'req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.973070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c96c719420c4597a5728951b75020e2', 'request_id': 'req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.986293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] The token cache is being invalidate because project 3c96c719420c4597a5728951b75020e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:24.986583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.987250 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:24.988879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1311/2630] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:24 2022] DELETE /identity/v3/projects/3c96c719420c4597a5728951b75020e2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:24.993735 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:25.005154 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4d61b-0da2-4695-831d-bf7d89625a0b service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:25.069887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:25.070772 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.071055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.071329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] PATH_INFO: `/v3/users/448a5ec686534e908de8ef2696e2002e` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.072891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] RBAC: Authorizing `identity:delete_user(user_id=448a5ec686534e908de8ef2696e2002e)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:25.074833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:25.123136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '448a5ec686534e908de8ef2696e2002e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.131169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '448a5ec686534e908de8ef2696e2002e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.138589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '448a5ec686534e908de8ef2696e2002e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.139367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '448a5ec686534e908de8ef2696e2002e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.140000 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-0937b185-92ea-4a82-859f-8235c0c9026e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '448a5ec686534e908de8ef2696e2002e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.159202 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1320/2631] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:24 2022] DELETE /identity/v3/users/448a5ec686534e908de8ef2696e2002e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:25.442114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:25.443621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:25.459642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49d0c5-1ce7-4f83-af6b-61cc8eada204 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:25.521453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:25.522383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.522669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.522941 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] PATH_INFO: `/v3/projects/6919ccaecee549da99603d9fc4008e30` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.524280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6919ccaecee549da99603d9fc4008e30)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:25.526341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:25.572688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6919ccaecee549da99603d9fc4008e30', 'request_id': 'req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.580149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6919ccaecee549da99603d9fc4008e30', 'request_id': 'req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.585495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6919ccaecee549da99603d9fc4008e30', 'request_id': 'req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.597452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] The token cache is being invalidate because project 6919ccaecee549da99603d9fc4008e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:25.597770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.598441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-1c311dd0-51b1-4ad6-9672-2fcab1f6413c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:25.599978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1312/2632] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:25 2022] DELETE /identity/v3/projects/6919ccaecee549da99603d9fc4008e30 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:25.605405 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7968ee-bfac-4230-bd43-b475d5f839f3 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.605693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7968ee-bfac-4230-bd43-b475d5f839f3 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.605991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7968ee-bfac-4230-bd43-b475d5f839f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.624890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-8c7968ee-bfac-4230-bd43-b475d5f839f3 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:25.626648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8c7968ee-bfac-4230-bd43-b475d5f839f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:25.667596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:25.689329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1321/2633] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:25.697070 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0cfdf-59a6-4e18-b140-df826df588f1 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.698087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0cfdf-59a6-4e18-b140-df826df588f1 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.698375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0cfdf-59a6-4e18-b140-df826df588f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.718572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-83f0cfdf-59a6-4e18-b140-df826df588f1 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:25.720316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-83f0cfdf-59a6-4e18-b140-df826df588f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:25.728523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1313/2634] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:25.732834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7968ee-bfac-4230-bd43-b475d5f839f3 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:25.743771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7968ee-bfac-4230-bd43-b475d5f839f3 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:25.748645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:25.749493 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.749661 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.749878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.750618 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:55:25.751010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:25.752612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:25.768874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1322/2635] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:55:25.772531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0cfdf-59a6-4e18-b140-df826df588f1 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:25.782877 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0cfdf-59a6-4e18-b140-df826df588f1 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:25.787693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:25.788369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.788687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.788968 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.789675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:25.791572 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:25.811802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:25.820157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1314/2636] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:25 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:55:25.825555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:25.837030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37e3a9-d73b-4da0-9323-06954f3b1941 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:25.902452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:25.903290 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.903440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.903738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.904348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:25.906442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:25.931388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1323/2637] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:25 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:25.932655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:25.944456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc953c5-8dbe-412d-812b-1e68dfa1ab51 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:25.950521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:25.951289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.951656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.951977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.952803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:25.955059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:25.969150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1315/2638] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:25.971049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:25.982812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36961180-f1ce-47eb-87b9-d74bc8bd2dfb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:25.987798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd42b-09be-4abd-ac5c-7512329ffec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:25.988472 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd42b-09be-4abd-ac5c-7512329ffec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:25.988798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd42b-09be-4abd-ac5c-7512329ffec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:25.989077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd42b-09be-4abd-ac5c-7512329ffec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:25.989810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd42b-09be-4abd-ac5c-7512329ffec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:25.991418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd42b-09be-4abd-ac5c-7512329ffec2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:26.002574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1324/2639] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:26.006794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:26.018416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019713b-ea40-4729-88fc-aadb4a7904b9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:26.023292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:26.024247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:26.024528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:26.024839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] PATH_INFO: `/v3/projects/673055bc8b3e4472a2d9e0ad973edabb/users/45b95b22fbb7424b97bc4e5adf67b1cd/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:26.048889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=673055bc8b3e4472a2d9e0ad973edabb, user_id=45b95b22fbb7424b97bc4e5adf67b1cd, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:26.051797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:26.067527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1316/2640] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:26 2022] PUT /identity/v3/projects/673055bc8b3e4472a2d9e0ad973edabb/users/45b95b22fbb7424b97bc4e5adf67b1cd/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:26.079071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:26.079522 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:26.079903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:26.107620 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] MFA Rules not processed for user `45b95b22fbb7424b97bc4e5adf67b1cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:26.108730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:26.184244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1325/2641] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:55:26.194656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:26.196399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:26.261016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac6264-7304-43ee-88d8-e0e0a5ae99e7 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:26.264900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:26.265677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:26.265864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:26.266206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:26.275653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:26.277299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:26.287154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1317/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:28.265452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:28.267227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:28.335713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:28.387679 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5f77e-2ee3-4ef6-a4b2-7cd9662c5781 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:28.395183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c753e52-efc6-4fbf-a340-03e798e91c14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:28.395868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c753e52-efc6-4fbf-a340-03e798e91c14 service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:28.396156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c753e52-efc6-4fbf-a340-03e798e91c14 service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:28.396440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c753e52-efc6-4fbf-a340-03e798e91c14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:28.398411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4c753e52-efc6-4fbf-a340-03e798e91c14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:28.423589 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c753e52-efc6-4fbf-a340-03e798e91c14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:28.426299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c753e52-efc6-4fbf-a340-03e798e91c14 service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:28.446630 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1326/2643] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:36.282336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:38.436327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=16 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:48.228867 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:48.298849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ec92-0e25-433b-9524-1c3f602503e8 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:48.305128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:48.306976 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:48.307666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:48.307903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] PATH_INFO: `/v3/users/45b95b22fbb7424b97bc4e5adf67b1cd` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:48.314052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b95b22fbb7424b97bc4e5adf67b1cd)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:48.316161 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:48.377598 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b95b22fbb7424b97bc4e5adf67b1cd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:48.386176 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b95b22fbb7424b97bc4e5adf67b1cd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:48.393427 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b95b22fbb7424b97bc4e5adf67b1cd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:48.394209 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b95b22fbb7424b97bc4e5adf67b1cd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:48.395576 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b95b22fbb7424b97bc4e5adf67b1cd'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:48.413689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1318/2644] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:48 2022] DELETE /identity/v3/users/45b95b22fbb7424b97bc4e5adf67b1cd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:48.418112 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:48.418593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:48.419248 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:48.438450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:48.439643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:48.498103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1327/2645] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:48.508385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:48.509853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:48.565922 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fab198-d7a7-47df-884a-7049024b71bd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:48.570164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b876f-50f5-4d92-9238-d79ceaacc22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:48.570804 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b876f-50f5-4d92-9238-d79ceaacc22f service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:48.571112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b876f-50f5-4d92-9238-d79ceaacc22f service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:48.571390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b876f-50f5-4d92-9238-d79ceaacc22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:48.580238 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b876f-50f5-4d92-9238-d79ceaacc22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:48.581685 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b876f-50f5-4d92-9238-d79ceaacc22f service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:48.591452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1319/2646] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:48.867144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:48.868811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:48.883117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72950b-fd57-4dc6-97d6-3960996bc72a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:48.909519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:48.910948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:48.911528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:48.911981 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] PATH_INFO: `/v3/projects/673055bc8b3e4472a2d9e0ad973edabb` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:48.914025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673055bc8b3e4472a2d9e0ad973edabb)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:48.916236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:48.983025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673055bc8b3e4472a2d9e0ad973edabb', 'request_id': 'req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:48.993318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673055bc8b3e4472a2d9e0ad973edabb', 'request_id': 'req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:49.000026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673055bc8b3e4472a2d9e0ad973edabb', 'request_id': 'req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:49.016822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] The token cache is being invalidate because project 673055bc8b3e4472a2d9e0ad973edabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:49.017383 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:49.018047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-b32ea33a-dda5-4dd0-bcc7-86a4ff8de6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:49.023941 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1328/2647] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:48 2022] DELETE /identity/v3/projects/673055bc8b3e4472a2d9e0ad973edabb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:49.025696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2af5-47fc-4e73-9068-28d0a6deff9a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.025985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2af5-47fc-4e73-9068-28d0a6deff9a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.026316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2af5-47fc-4e73-9068-28d0a6deff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.056112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-7f7f2af5-47fc-4e73-9068-28d0a6deff9a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:49.058075 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7f7f2af5-47fc-4e73-9068-28d0a6deff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:49.090739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:49.170231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1320/2648] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:49.179259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a2b5-499b-47ac-a22a-d3257e1e7083 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.179553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a2b5-499b-47ac-a22a-d3257e1e7083 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.179837 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a2b5-499b-47ac-a22a-d3257e1e7083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.199951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6b91a2b5-499b-47ac-a22a-d3257e1e7083 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:49.201119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6b91a2b5-499b-47ac-a22a-d3257e1e7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:49.209429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1329/2649] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:49.213789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2af5-47fc-4e73-9068-28d0a6deff9a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:49.223866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2af5-47fc-4e73-9068-28d0a6deff9a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:49.228467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:49.229151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.229797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.229797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.230459 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:55:49.230761 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:49.232348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:49.244907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1321/2650] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:55:49.249318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91a2b5-499b-47ac-a22a-d3257e1e7083 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:49.258820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91a2b5-499b-47ac-a22a-d3257e1e7083 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:49.263428 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:49.264115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.264381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.264689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.265381 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:49.267270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:49.294269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1330/2651] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:55:49 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:49.297945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:49.308757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b5681-3409-47b0-98ae-7fb4dea2ff04 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:49.365471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:49.366142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.366428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.366710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.367407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:49.369487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:49.389370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1322/2652] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:55:49 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:55:49.393571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:49.404903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba628bd-5dd0-4507-9251-0195d0c91031 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:49.409675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:49.410950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.411259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.411531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.412258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:49.414195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:49.424857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1331/2653] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:49.429111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:49.439239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c49a3-c53e-406d-a70f-2369700d4a7d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:49.443924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84843e-aa2b-4926-a056-c049f1cea473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:49.444594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84843e-aa2b-4926-a056-c049f1cea473 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.444887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84843e-aa2b-4926-a056-c049f1cea473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.445152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84843e-aa2b-4926-a056-c049f1cea473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.445870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84843e-aa2b-4926-a056-c049f1cea473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:49.447441 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84843e-aa2b-4926-a056-c049f1cea473 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:49.457933 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1323/2654] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:55:49.461695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:49.471505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25673fd-4f25-4a89-b157-c63f65b15b55 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:49.476834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:49.477696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.477982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.478258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] PATH_INFO: `/v3/projects/3d7cd74c906c4307ae9a7b36a763ba3d/users/066086bfb0f142978c8493b64e55dc15/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.492575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7cd74c906c4307ae9a7b36a763ba3d, user_id=066086bfb0f142978c8493b64e55dc15, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:49.495207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:49.507849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1332/2655] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:55:49 2022] PUT /identity/v3/projects/3d7cd74c906c4307ae9a7b36a763ba3d/users/066086bfb0f142978c8493b64e55dc15/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:49.512829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e5957-e58e-49f2-a6e7-5a2795928f8a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.513152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e5957-e58e-49f2-a6e7-5a2795928f8a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.513453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e5957-e58e-49f2-a6e7-5a2795928f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.531793 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e91e5957-e58e-49f2-a6e7-5a2795928f8a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:49.533037 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e91e5957-e58e-49f2-a6e7-5a2795928f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:49.591367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1324/2656] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:49.601350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:49.603027 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:49.625970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:49.664228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42226eb-a5c5-4b2a-8cbb-fcd8e908b3b9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:49.667896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:49.668536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:49.668847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:49.669121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:49.678435 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:49.679849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:49.689324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1333/2657] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:50.279077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878156-f141-4c53-a5e9-c1dd37d047e9 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:50.279640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878156-f141-4c53-a5e9-c1dd37d047e9 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:50.279640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878156-f141-4c53-a5e9-c1dd37d047e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:50.306651 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-82878156-f141-4c53-a5e9-c1dd37d047e9 None None] MFA Rules not processed for user `066086bfb0f142978c8493b64e55dc15`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:50.307946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-82878156-f141-4c53-a5e9-c1dd37d047e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:50.328599 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:50.377725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1325/2658] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:50.388961 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:50.390537 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:50.451146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06d30a-1e92-416d-b2d5-6692e31e1f4e service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:50.454905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:50.455555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:50.455836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:50.456115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:50.465068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:50.466496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:50.475925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1334/2659] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:54.959606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:54.960168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:54.960168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:54.978167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] MFA Rules not processed for user `066086bfb0f142978c8493b64e55dc15`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:55:54.979364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:54.986188 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1326/2660] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:55:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:55:55.215291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:55.216728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:55.229433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597178f5-819b-4a7c-819f-750e438a5f3d service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:55.233033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:55.233665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:55.233947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:55.234253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:55.235955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:55.246059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:55.247469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:55.255648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1335/2661] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:59.904446 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:59.906096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:59.988391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e22887-4786-46af-b771-e19b66c7bef5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:59.994061 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4de283-5357-4a88-9bd6-dc9dffb41eda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:59.994726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4de283-5357-4a88-9bd6-dc9dffb41eda service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:59.995011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4de283-5357-4a88-9bd6-dc9dffb41eda service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:59.995291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4de283-5357-4a88-9bd6-dc9dffb41eda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:59.997107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0c4de283-5357-4a88-9bd6-dc9dffb41eda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:00.009894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4de283-5357-4a88-9bd6-dc9dffb41eda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:00.011561 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4de283-5357-4a88-9bd6-dc9dffb41eda service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:00.023274 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1327/2662] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:05.251821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=49 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:09.444255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:09.446194 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:09.461830 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b542a-eda0-407e-95b8-1ce1202eca64 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:09.467821 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:09.468725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:09.469010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:09.469280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] PATH_INFO: `/v3/users/3900f2a435af4626b1b9c39dc9c547b7` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:09.470488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3900f2a435af4626b1b9c39dc9c547b7)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:09.472414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:09.524695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3900f2a435af4626b1b9c39dc9c547b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.532534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3900f2a435af4626b1b9c39dc9c547b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.540153 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3900f2a435af4626b1b9c39dc9c547b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.540842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3900f2a435af4626b1b9c39dc9c547b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.541444 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3900f2a435af4626b1b9c39dc9c547b7'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.559719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1336/2663] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:09 2022] DELETE /identity/v3/users/3900f2a435af4626b1b9c39dc9c547b7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:09.564227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:09.564717 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:09.564799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:09.584458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:09.585615 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:09.644468 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1328/2664] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:09.654672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:09.656086 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:09.715042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d5c31-6ea7-43c6-994b-5f5eeaf3f405 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:09.718749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2edc31-2859-4cb4-b3c0-072293bc3368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:09.719407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2edc31-2859-4cb4-b3c0-072293bc3368 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:09.719689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2edc31-2859-4cb4-b3c0-072293bc3368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:09.719972 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2edc31-2859-4cb4-b3c0-072293bc3368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:09.730395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2edc31-2859-4cb4-b3c0-072293bc3368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:09.731863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2edc31-2859-4cb4-b3c0-072293bc3368 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:09.743252 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1337/2665] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:56:09.758779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:09.760240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:09.775610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41166358-e01e-42da-a2e9-ba836119f709 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:09.780367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:09.781477 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:09.781751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:09.782042 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] PATH_INFO: `/v3/users/2e0e6c1ad20a48a38b9e9c5b446334fb` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:09.783382 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e0e6c1ad20a48a38b9e9c5b446334fb)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:09.785386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:09.831507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:09.847083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e0e6c1ad20a48a38b9e9c5b446334fb'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.855842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e0e6c1ad20a48a38b9e9c5b446334fb'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.864733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e0e6c1ad20a48a38b9e9c5b446334fb'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.865521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e0e6c1ad20a48a38b9e9c5b446334fb'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.866213 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e0e6c1ad20a48a38b9e9c5b446334fb'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:09.884618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1329/2666] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:09 2022] DELETE /identity/v3/users/2e0e6c1ad20a48a38b9e9c5b446334fb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:09.889129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:09.889449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:09.889731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:09.908203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:09.909457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:09.979644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1338/2667] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:09.991348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:09.993185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.071911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44ee84-23a7-48a0-ae4c-7cfcf9dc2825 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:10.084910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:10.084910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.084910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.084910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.085878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:10.087577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.094663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:10.097421 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:10.100789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1834f-d330-48d5-b5ee-08a3638bbcf8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:10.106533 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:10.107515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.107713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.107991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] PATH_INFO: `/v3/projects/b2877ec8c29249ca83d2e75dad7d13be` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.109270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2877ec8c29249ca83d2e75dad7d13be)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:10.111207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:10.122727 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1330/2668] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:10.127812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:10.177820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2877ec8c29249ca83d2e75dad7d13be', 'request_id': 'req-44d40ced-6c00-48e5-be99-825bbef1bc4e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.186778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2877ec8c29249ca83d2e75dad7d13be', 'request_id': 'req-44d40ced-6c00-48e5-be99-825bbef1bc4e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.197284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2877ec8c29249ca83d2e75dad7d13be', 'request_id': 'req-44d40ced-6c00-48e5-be99-825bbef1bc4e'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.210310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] The token cache is being invalidate because project b2877ec8c29249ca83d2e75dad7d13be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:10.210602 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.211279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.217107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1339/2669] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:10 2022] DELETE /identity/v3/projects/b2877ec8c29249ca83d2e75dad7d13be => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:10.230904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:10.241479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f0efc-d112-4413-9d2f-29ce97399841 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:10.323646 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:10.324536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.324846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.325121 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] PATH_INFO: `/v3/users/ffe9120727ad446e90023ce2d7eb231c` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.326365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe9120727ad446e90023ce2d7eb231c)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:10.328458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:10.381369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe9120727ad446e90023ce2d7eb231c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.390313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe9120727ad446e90023ce2d7eb231c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.391677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:10.400335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe9120727ad446e90023ce2d7eb231c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.401276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe9120727ad446e90023ce2d7eb231c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.402334 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d2b0f616-9be7-44cb-86a2-19a28dd61ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe9120727ad446e90023ce2d7eb231c'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.412360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:10.413957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.431186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1331/2670] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:10 2022] DELETE /identity/v3/users/ffe9120727ad446e90023ce2d7eb231c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:10.433799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d40ced-6c00-48e5-be99-825bbef1bc4e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:10.498067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:10.498962 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.499244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.499517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] PATH_INFO: `/v3/projects/fe625d7fd44a439683fc1d2bc4cd6cb5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.500796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe625d7fd44a439683fc1d2bc4cd6cb5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:10.502751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:10.558650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe625d7fd44a439683fc1d2bc4cd6cb5', 'request_id': 'req-182e94c4-534d-4e38-838b-4c5913f178dc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.572167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe625d7fd44a439683fc1d2bc4cd6cb5', 'request_id': 'req-182e94c4-534d-4e38-838b-4c5913f178dc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.578742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe625d7fd44a439683fc1d2bc4cd6cb5', 'request_id': 'req-182e94c4-534d-4e38-838b-4c5913f178dc'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.595366 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] The token cache is being invalidate because project fe625d7fd44a439683fc1d2bc4cd6cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:10.595665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.596357 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.598026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1340/2671] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:10 2022] DELETE /identity/v3/projects/fe625d7fd44a439683fc1d2bc4cd6cb5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:10.603128 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4cbc9-8552-4462-b7ad-e6fe8f9f711e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.603837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4cbc9-8552-4462-b7ad-e6fe8f9f711e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.603837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4cbc9-8552-4462-b7ad-e6fe8f9f711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.631569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-55b4cbc9-8552-4462-b7ad-e6fe8f9f711e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:10.632982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-55b4cbc9-8552-4462-b7ad-e6fe8f9f711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.709016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1332/2672] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:10.734003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:10.737109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.761213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e94c4-534d-4e38-838b-4c5913f178dc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:10.767921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:10.768836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.769118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.769391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] PATH_INFO: `/v3/projects/1f759e9436d4497f8ef6a8ecc4f604b2` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.771277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f759e9436d4497f8ef6a8ecc4f604b2)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:10.773325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:10.778340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f6221-e364-48f5-9056-772354518bee None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.779196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f6221-e364-48f5-9056-772354518bee None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.779196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f6221-e364-48f5-9056-772354518bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.805855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-0b6f6221-e364-48f5-9056-772354518bee None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:10.809969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:10.810454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0b6f6221-e364-48f5-9056-772354518bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.833975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1333/2673] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:10.835620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f6221-e364-48f5-9056-772354518bee None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:10.845868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f6221-e364-48f5-9056-772354518bee None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:10.854573 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f759e9436d4497f8ef6a8ecc4f604b2', 'request_id': 'req-4cd70b66-97c8-4026-9b55-60f2019fa23b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.869168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f759e9436d4497f8ef6a8ecc4f604b2', 'request_id': 'req-4cd70b66-97c8-4026-9b55-60f2019fa23b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.881703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f759e9436d4497f8ef6a8ecc4f604b2', 'request_id': 'req-4cd70b66-97c8-4026-9b55-60f2019fa23b'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.898279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] The token cache is being invalidate because project 1f759e9436d4497f8ef6a8ecc4f604b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:10.898877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.899586 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4cd70b66-97c8-4026-9b55-60f2019fa23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:10.906006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1341/2674] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:10 2022] DELETE /identity/v3/projects/1f759e9436d4497f8ef6a8ecc4f604b2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:10.913685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75dffc-2a96-4e31-9ca6-12412cf9b40b None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.914144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75dffc-2a96-4e31-9ca6-12412cf9b40b None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.914628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75dffc-2a96-4e31-9ca6-12412cf9b40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.935703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:10.937842 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.938483 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.939118 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.941156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:56:10.942749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:10.945934 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-ad75dffc-2a96-4e31-9ca6-12412cf9b40b None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:10.946185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:10.948005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ad75dffc-2a96-4e31-9ca6-12412cf9b40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.968916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1342/2675] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:10.970588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3f82b-92cd-4db5-af7e-4591e036aed7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:10.970959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3f82b-92cd-4db5-af7e-4591e036aed7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:10.971564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3f82b-92cd-4db5-af7e-4591e036aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:10.975146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1334/2676] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:56:10.979600 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:10.981551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:10.995856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-bcf3f82b-92cd-4db5-af7e-4591e036aed7 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:10.997015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e3305-a511-490f-9110-2a9ee20871b4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:10.997183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bcf3f82b-92cd-4db5-af7e-4591e036aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:11.001944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.002607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.002889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.003167 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.003860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.008754 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1343/2677] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:11.008920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.011759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3f82b-92cd-4db5-af7e-4591e036aed7 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.032285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3f82b-92cd-4db5-af7e-4591e036aed7 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.037947 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.038716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.039004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.039285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.040063 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:56:11.040389 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.042513 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.058082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1344/2678] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:56:11.074039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.074716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1335/2679] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:10 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:11.078189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.096037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ee97-40cb-47e7-91af-96d63939fceb admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.143283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:11.179134 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.179920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.180088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.180478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.181084 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.183307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.184057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cca0f4-6406-4f2c-9080-7afb7f71eb73 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.191403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.192539 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.193157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.193632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.196008 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.198047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.244043 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1336/2680] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:11 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:11.246441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1345/2681] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:11 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:11.250137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.251382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.262071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897e59-3837-4c3c-b828-e6349acff252 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.342111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.342872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.343159 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.343434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.344195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.346013 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.347285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8f928-6d14-4d07-9324-ae170624c0ec admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.352918 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.353603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.353957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.354237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.354932 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.356706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1337/2682] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:11.357888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.360636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.371194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038a68e-522d-4cdb-ba3c-a46b1b8288bb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.376450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.377145 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.377431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.377707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.378433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.380582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:11.380692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.390225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1346/2683] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:11 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:56:11.391320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.392403 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1338/2684] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:56:11.396511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.402566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59ae57-a73d-41a0-b4e7-b0391eddf74a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.406240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafe0a0-00c3-45c3-b932-4de96320441b admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.407247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.407914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.408240 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.408470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.409239 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.411474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.412222 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfc651-f6e5-44ee-95e0-1899793d9362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.413306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc651-f6e5-44ee-95e0-1899793d9362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.413731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc651-f6e5-44ee-95e0-1899793d9362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.414150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfc651-f6e5-44ee-95e0-1899793d9362 admin admin] PATH_INFO: `/v3/projects/49df37ab58464b9dbb9d79d8a6ba5c23/users/cf3828054d1647db99c8daf321c5c398/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.422935 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1347/2685] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:11.428788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.431493 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfc651-f6e5-44ee-95e0-1899793d9362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49df37ab58464b9dbb9d79d8a6ba5c23, user_id=cf3828054d1647db99c8daf321c5c398, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.434495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfc651-f6e5-44ee-95e0-1899793d9362 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.440362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a12036-353a-479e-8c6c-b8f7943b54d5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.446761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.447801 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.448015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.448295 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.450089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.450207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1339/2686] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:11 2022] PUT /identity/v3/projects/49df37ab58464b9dbb9d79d8a6ba5c23/users/cf3828054d1647db99c8daf321c5c398/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:11.450997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.461153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.461454 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.461720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.464715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1348/2687] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:56:11.467448 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.479125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eff765-5b4d-4254-8f3a-680f8621aa97 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.492367 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] MFA Rules not processed for user `cf3828054d1647db99c8daf321c5c398`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:11.493628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:11.554647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e625c-f52c-401c-805e-d9b3c28f43e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.555727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e625c-f52c-401c-805e-d9b3c28f43e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.555966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e625c-f52c-401c-805e-d9b3c28f43e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.556738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e625c-f52c-401c-805e-d9b3c28f43e5 admin admin] PATH_INFO: `/v3/projects/54e78fcc269c4420b1fd408741b069bf/users/95626fb9357e42929f45e1055441688b/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.569992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1340/2688] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:11.572471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e625c-f52c-401c-805e-d9b3c28f43e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e78fcc269c4420b1fd408741b069bf, user_id=95626fb9357e42929f45e1055441688b, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.574948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e625c-f52c-401c-805e-d9b3c28f43e5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.582174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.584020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:11.591249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1349/2689] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:11 2022] PUT /identity/v3/projects/54e78fcc269c4420b1fd408741b069bf/users/95626fb9357e42929f45e1055441688b/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:11.601713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f2f63-43dd-4a0b-a0de-316886a33462 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.602058 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f2f63-43dd-4a0b-a0de-316886a33462 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.602341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f2f63-43dd-4a0b-a0de-316886a33462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.605316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:11.629719 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-562f2f63-43dd-4a0b-a0de-316886a33462 None None] MFA Rules not processed for user `95626fb9357e42929f45e1055441688b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:11.630997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-562f2f63-43dd-4a0b-a0de-316886a33462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:11.646848 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2bc7f-af48-4069-8ca4-e7fd2516b6d5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.705659 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1350/2690] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:11.710429 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f2f63-43dd-4a0b-a0de-316886a33462 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.714650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.715312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.715587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.715859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.768833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:11.783675 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f2f63-43dd-4a0b-a0de-316886a33462 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.787348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.788575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.788999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.789476 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.789877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.790210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.791045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.792800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.801892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1341/2691] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:56:11.822229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1351/2692] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:11 2022] POST /identity/v3/projects => generated 401 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:56:11.826348 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.837905 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f662-9b62-450c-9b81-a955c3bcbc1f service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.917450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.918303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.918668 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.919025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.919838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.922821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.949246 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1342/2693] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:11 2022] POST /identity/v3/users => generated 307 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:11.950500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:11.962485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d20141-9fd4-467e-a433-1364bdc839e2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:11.967682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:11.968422 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:11.968873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:11.969165 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:11.970146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:11.972293 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:11.984763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1352/2694] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:56:11.989610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:12.001707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18351fdf-3487-4fd2-900c-03469ee662fb admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:12.007107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:12.008136 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:12.008471 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:12.008780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] PATH_INFO: `/v3/projects/16daa57c63404626a763ff8939280c72/users/eee128a805574bd1b3dc217392f170af/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:12.027087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=16daa57c63404626a763ff8939280c72, user_id=eee128a805574bd1b3dc217392f170af, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:12.029827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:12.033228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:12.049001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1343/2695] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:11 2022] PUT /identity/v3/projects/16daa57c63404626a763ff8939280c72/users/eee128a805574bd1b3dc217392f170af/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:12.053796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:12.064451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30825656-acd1-4058-9840-76cb1c8ac95d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:12.130964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794e9ed-9c2d-44d6-a295-6dba3778a5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:12.131724 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e9ed-9c2d-44d6-a295-6dba3778a5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:12.132088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e9ed-9c2d-44d6-a295-6dba3778a5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:12.132434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e9ed-9c2d-44d6-a295-6dba3778a5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:12.133269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794e9ed-9c2d-44d6-a295-6dba3778a5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:12.134967 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794e9ed-9c2d-44d6-a295-6dba3778a5a3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:12.146584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1353/2696] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:12.150736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:12.162036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eba24c-2ac5-42b9-9436-7ea89b48acbd admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:12.167551 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:12.168753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:12.169035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:12.169307 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] PATH_INFO: `/v3/projects/16daa57c63404626a763ff8939280c72/users/eee128a805574bd1b3dc217392f170af/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:12.171447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16daa57c63404626a763ff8939280c72, user_id=eee128a805574bd1b3dc217392f170af, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:12.175091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:12.188206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1344/2697] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:12 2022] PUT /identity/v3/projects/16daa57c63404626a763ff8939280c72/users/eee128a805574bd1b3dc217392f170af/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:12.200518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830781-5a1d-49c6-b0ea-0212d29578ba None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:12.200903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830781-5a1d-49c6-b0ea-0212d29578ba None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:12.201213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830781-5a1d-49c6-b0ea-0212d29578ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:12.229459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-dc830781-5a1d-49c6-b0ea-0212d29578ba None None] MFA Rules not processed for user `eee128a805574bd1b3dc217392f170af`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:12.230707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-dc830781-5a1d-49c6-b0ea-0212d29578ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:12.304361 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1354/2698] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:12.315524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:12.376521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47210-48ae-4e95-a234-f09c4e4cf424 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:12.381184 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:12.381846 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:12.382126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:12.382405 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:12.393276 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:12.394760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:12.406622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1345/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:12 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:12.485905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc830781-5a1d-49c6-b0ea-0212d29578ba None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:12.496343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc830781-5a1d-49c6-b0ea-0212d29578ba None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:12.500572 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141e418-0e01-4d0e-88f7-c1f0df53aa0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:12.501253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e418-0e01-4d0e-88f7-c1f0df53aa0f service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:12.501535 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e418-0e01-4d0e-88f7-c1f0df53aa0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:12.501799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e418-0e01-4d0e-88f7-c1f0df53aa0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:12.548118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:12.563046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141e418-0e01-4d0e-88f7-c1f0df53aa0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:12.564470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141e418-0e01-4d0e-88f7-c1f0df53aa0f service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:12.574289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1355/2700] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:19.828862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:19.899023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664f721-074e-4469-8431-a6df27333eee service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:19.903640 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:19.904521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:19.904840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:19.905119 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] PATH_INFO: `/v3/users/95626fb9357e42929f45e1055441688b` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:19.906507 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95626fb9357e42929f45e1055441688b)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:19.908424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:19.961949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95626fb9357e42929f45e1055441688b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:19.970414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95626fb9357e42929f45e1055441688b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:19.978594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95626fb9357e42929f45e1055441688b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:19.979291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95626fb9357e42929f45e1055441688b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:19.979904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95626fb9357e42929f45e1055441688b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:19.997962 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1346/2701] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:19 2022] DELETE /identity/v3/users/95626fb9357e42929f45e1055441688b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:20.002828 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:20.003286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:20.003388 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:20.021331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:20.023076 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:20.087952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1356/2702] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:20.097533 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:20.098911 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:20.105182 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:20.159060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc46df7-de94-4a5e-ad58-a4f303224104 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:20.163570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:20.164224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:20.164508 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:20.164819 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:20.174096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:20.175784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:20.185385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1347/2703] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:20.457523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:20.459623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:20.473812 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c5fec-e258-4e83-98de-6937d4c8c7fe None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:20.479938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:20.480848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:20.481137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:20.481414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] PATH_INFO: `/v3/projects/54e78fcc269c4420b1fd408741b069bf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:20.483009 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e78fcc269c4420b1fd408741b069bf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:20.484762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:20.536649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e78fcc269c4420b1fd408741b069bf', 'request_id': 'req-a1af4761-df8c-4b5f-bcca-0505e0d0d415'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.545658 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e78fcc269c4420b1fd408741b069bf', 'request_id': 'req-a1af4761-df8c-4b5f-bcca-0505e0d0d415'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.551905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e78fcc269c4420b1fd408741b069bf', 'request_id': 'req-a1af4761-df8c-4b5f-bcca-0505e0d0d415'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.565852 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] The token cache is being invalidate because project 54e78fcc269c4420b1fd408741b069bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:20.566148 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.566896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.568524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1357/2704] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:20 2022] DELETE /identity/v3/projects/54e78fcc269c4420b1fd408741b069bf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:20.572578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:20.582277 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd56f91-852e-49e6-a348-dd5495eb735d service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:20.643702 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:20.644603 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:20.644893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:20.645194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] PATH_INFO: `/v3/users/eee128a805574bd1b3dc217392f170af` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:20.646373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee128a805574bd1b3dc217392f170af)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:20.648923 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:20.696644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee128a805574bd1b3dc217392f170af'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.704125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee128a805574bd1b3dc217392f170af'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.711029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee128a805574bd1b3dc217392f170af'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.711684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee128a805574bd1b3dc217392f170af'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.712280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-e8c77957-1431-463c-8929-1753a444fcc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee128a805574bd1b3dc217392f170af'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:20.729788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1348/2705] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:20 2022] DELETE /identity/v3/users/eee128a805574bd1b3dc217392f170af => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:21.004192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.005751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:21.019012 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af4761-df8c-4b5f-bcca-0505e0d0d415 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.089460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.090323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.090611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.090893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] PATH_INFO: `/v3/projects/16daa57c63404626a763ff8939280c72` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.092086 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=16daa57c63404626a763ff8939280c72)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.094042 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.104925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:21.143016 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16daa57c63404626a763ff8939280c72', 'request_id': 'req-5486c71a-2879-4037-9fdc-17501947f7fe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:21.150527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16daa57c63404626a763ff8939280c72', 'request_id': 'req-5486c71a-2879-4037-9fdc-17501947f7fe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:21.156139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16daa57c63404626a763ff8939280c72', 'request_id': 'req-5486c71a-2879-4037-9fdc-17501947f7fe'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:21.167871 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] The token cache is being invalidate because project 16daa57c63404626a763ff8939280c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:21.168162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:21.168847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5486c71a-2879-4037-9fdc-17501947f7fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:21.170398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1358/2706] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:21 2022] DELETE /identity/v3/projects/16daa57c63404626a763ff8939280c72 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:21.175509 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b64f-11b1-4b9d-82fc-89ca46db4e70 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.175978 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b64f-11b1-4b9d-82fc-89ca46db4e70 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.176057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b64f-11b1-4b9d-82fc-89ca46db4e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.194146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-6b91b64f-11b1-4b9d-82fc-89ca46db4e70 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:21.195294 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-6b91b64f-11b1-4b9d-82fc-89ca46db4e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:21.214371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:21.260320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1349/2707] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:21.268109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ce5f9-5252-46cb-b926-de4ea05587ba None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.268397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ce5f9-5252-46cb-b926-de4ea05587ba None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.268740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ce5f9-5252-46cb-b926-de4ea05587ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.286795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f37ce5f9-5252-46cb-b926-de4ea05587ba None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:21.288596 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f37ce5f9-5252-46cb-b926-de4ea05587ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:21.296369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1359/2708] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:21.300864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91b64f-11b1-4b9d-82fc-89ca46db4e70 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.311339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91b64f-11b1-4b9d-82fc-89ca46db4e70 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.315982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.316672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.316957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.317233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.317958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:56:21.318312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.319913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.331757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1350/2709] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:56:21.336029 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ce5f9-5252-46cb-b926-de4ea05587ba None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.345378 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ce5f9-5252-46cb-b926-de4ea05587ba None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.350496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.351362 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.351749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.352026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.352751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.354639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.382142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1360/2710] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:21 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:21.386103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.395980 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ef37d-716d-4456-a243-9819f08c121c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.453231 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.453824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.454111 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.454432 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.455131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.457059 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.478053 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1351/2711] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:21 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:21.482228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.493159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104c9df-6c01-4410-b163-2d7754e61ddc admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.497743 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.498397 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.498680 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.498957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.499682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.501318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.512684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1361/2712] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:21.516953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.527550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66855aa6-bcfc-4d07-a29c-bcb4e6f6f0c0 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.532496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e586f9-48b4-4452-a99e-c67d4333ef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.533191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e586f9-48b4-4452-a99e-c67d4333ef89 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.533479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e586f9-48b4-4452-a99e-c67d4333ef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.533760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e586f9-48b4-4452-a99e-c67d4333ef89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.534495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e586f9-48b4-4452-a99e-c67d4333ef89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.536126 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e586f9-48b4-4452-a99e-c67d4333ef89 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.547077 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1352/2713] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:56:21.551450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.561648 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d442f-010f-4f19-b8e2-acb67f911399 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.566284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158419be-393e-4602-9475-c22b73262205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.567157 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158419be-393e-4602-9475-c22b73262205 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.567440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158419be-393e-4602-9475-c22b73262205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.567715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158419be-393e-4602-9475-c22b73262205 admin admin] PATH_INFO: `/v3/projects/d7fd3eafe3e246ffa5e2e4f6b0095307/users/06d115560faa4374922012bf85fb0515/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.583341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158419be-393e-4602-9475-c22b73262205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7fd3eafe3e246ffa5e2e4f6b0095307, user_id=06d115560faa4374922012bf85fb0515, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.586534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158419be-393e-4602-9475-c22b73262205 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.599159 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1362/2714] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:21 2022] PUT /identity/v3/projects/d7fd3eafe3e246ffa5e2e4f6b0095307/users/06d115560faa4374922012bf85fb0515/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:21.609913 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e472d-6395-44cc-8ca6-f1938ffccbc1 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.610206 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e472d-6395-44cc-8ca6-f1938ffccbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.610490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e472d-6395-44cc-8ca6-f1938ffccbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.638376 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-f62e472d-6395-44cc-8ca6-f1938ffccbc1 None None] MFA Rules not processed for user `06d115560faa4374922012bf85fb0515`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:21.639630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f62e472d-6395-44cc-8ca6-f1938ffccbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:21.704618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1353/2715] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:21.709518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158419be-393e-4602-9475-c22b73262205 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.769763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158419be-393e-4602-9475-c22b73262205 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.774991 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.775643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.775926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.776199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.776922 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.778824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.806338 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1363/2716] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:21 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:21.810472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e472d-6395-44cc-8ca6-f1938ffccbc1 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.821585 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e472d-6395-44cc-8ca6-f1938ffccbc1 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.839314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:21.882430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.883101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.883375 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.883641 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.884331 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.886264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.907767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1354/2717] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:21 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:21.911771 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.922044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bb220-21d8-45d3-af48-70edbf5ecddf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.926897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.927551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.927833 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.928107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.928874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.930485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.941200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1364/2718] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:21.945852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.955822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814ae82-daad-49f2-82db-562a7a12a8e4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.961495 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aa8d2-fe74-473b-adae-1ad55cd54f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.962038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa8d2-fe74-473b-adae-1ad55cd54f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.962320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa8d2-fe74-473b-adae-1ad55cd54f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.962597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa8d2-fe74-473b-adae-1ad55cd54f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.963320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aa8d2-fe74-473b-adae-1ad55cd54f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:21.964937 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aa8d2-fe74-473b-adae-1ad55cd54f1a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:21.975864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1355/2719] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:21.980051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:21.990136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f95c8c-a590-4254-b38a-ec2265a4f0bd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:21.995365 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:21.996237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:21.996518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:21.996823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] PATH_INFO: `/v3/projects/4dbbe2860ee74ce39761e5ea7c0b0605/users/85e0c3d974bf43cf9281cadbce79c7ca/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:21.999815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:22.013426 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dbbe2860ee74ce39761e5ea7c0b0605, user_id=85e0c3d974bf43cf9281cadbce79c7ca, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.016057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.029839 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1365/2720] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:21 2022] PUT /identity/v3/projects/4dbbe2860ee74ce39761e5ea7c0b0605/users/85e0c3d974bf43cf9281cadbce79c7ca/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:22.042101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7c528-fdf5-4705-a29b-6c8e231c0574 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.042397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7c528-fdf5-4705-a29b-6c8e231c0574 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.042708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7c528-fdf5-4705-a29b-6c8e231c0574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.069796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-fed7c528-fdf5-4705-a29b-6c8e231c0574 None None] MFA Rules not processed for user `85e0c3d974bf43cf9281cadbce79c7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:22.071011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-fed7c528-fdf5-4705-a29b-6c8e231c0574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:22.141645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1356/2721] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:22.145908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.208593 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f795e4a-b1c2-4d00-9304-5d958fe52a57 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.213322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.213975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.214253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.214526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.215220 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.217147 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.243646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1366/2722] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:22 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:22.247844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7c528-fdf5-4705-a29b-6c8e231c0574 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.258618 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7c528-fdf5-4705-a29b-6c8e231c0574 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.315324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.315989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.316269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.316543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.317262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.319858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.342306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1357/2723] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:22 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:22.346707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.357998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aee21-8eda-41e1-8435-2e9537b0b7a7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.362685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.363342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.363625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.363903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.364657 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.366250 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.377068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1367/2724] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:22.381285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.391871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5f3c7-3411-4ca9-aee6-0a3fe57e1946 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.397123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.397935 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.398212 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.398485 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] PATH_INFO: `/v3/projects/4e4644bcfa3048549766acecddf822e1/users/2b6eb1987bee4f65a05ead476b9c544b/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.414048 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4644bcfa3048549766acecddf822e1, user_id=2b6eb1987bee4f65a05ead476b9c544b, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.416773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.418998 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:22.430986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1358/2725] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:22 2022] PUT /identity/v3/projects/4e4644bcfa3048549766acecddf822e1/users/2b6eb1987bee4f65a05ead476b9c544b/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:22.435317 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.445819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58762594-617b-45a3-967b-000663bc0a6c admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.507603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e14cde-69c2-4ef7-b34f-647a02f08ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.508192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14cde-69c2-4ef7-b34f-647a02f08ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.508474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14cde-69c2-4ef7-b34f-647a02f08ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.508791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e14cde-69c2-4ef7-b34f-647a02f08ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.509625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e14cde-69c2-4ef7-b34f-647a02f08ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.511276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e14cde-69c2-4ef7-b34f-647a02f08ce2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.521096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1368/2726] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:22.525516 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.544480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112cf24-6ea3-45be-984e-cdfaee6174d5 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.549162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.550051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.550594 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.550870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] PATH_INFO: `/v3/projects/4e4644bcfa3048549766acecddf822e1/users/2b6eb1987bee4f65a05ead476b9c544b/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.553350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4644bcfa3048549766acecddf822e1, user_id=2b6eb1987bee4f65a05ead476b9c544b, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.555976 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.566358 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1359/2727] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:22 2022] PUT /identity/v3/projects/4e4644bcfa3048549766acecddf822e1/users/2b6eb1987bee4f65a05ead476b9c544b/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:22.577560 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d721584-2faa-4770-8e60-bd3ac1f2435e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.577847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d721584-2faa-4770-8e60-bd3ac1f2435e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.578126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d721584-2faa-4770-8e60-bd3ac1f2435e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.604770 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-2d721584-2faa-4770-8e60-bd3ac1f2435e None None] MFA Rules not processed for user `2b6eb1987bee4f65a05ead476b9c544b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:22.605996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-2d721584-2faa-4770-8e60-bd3ac1f2435e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:22.653703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:22.680045 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1369/2728] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:22.691352 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.693117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:22.750777 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ce14-b63e-40ae-8b08-9ee3d51482f1 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.754466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.755108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.755394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.755673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.765269 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.766634 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.776873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1360/2729] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:22.814944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d721584-2faa-4770-8e60-bd3ac1f2435e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:22.825267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d721584-2faa-4770-8e60-bd3ac1f2435e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:22.828847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614c4b5-1fa4-4537-b1b7-578775c00662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:22.829479 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614c4b5-1fa4-4537-b1b7-578775c00662 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:22.829762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614c4b5-1fa4-4537-b1b7-578775c00662 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:22.830034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614c4b5-1fa4-4537-b1b7-578775c00662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:22.886374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614c4b5-1fa4-4537-b1b7-578775c00662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:22.887800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614c4b5-1fa4-4537-b1b7-578775c00662 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:22.897382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1370/2730] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:32.105328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:32.168122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbe00-475c-4815-b54b-4bd7fc8d7f37 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:32.172681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:32.173582 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:32.173861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:32.174139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] PATH_INFO: `/v3/users/06d115560faa4374922012bf85fb0515` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:32.175601 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d115560faa4374922012bf85fb0515)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:32.177839 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:32.227930 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d115560faa4374922012bf85fb0515'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:32.235313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d115560faa4374922012bf85fb0515'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:32.242861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d115560faa4374922012bf85fb0515'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:32.243497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d115560faa4374922012bf85fb0515'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:32.244207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d115560faa4374922012bf85fb0515'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:32.264033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1361/2731] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:32 2022] DELETE /identity/v3/users/06d115560faa4374922012bf85fb0515 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:32.268431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:32.268924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:32.269127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:32.287668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:32.288908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:32.358740 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1371/2732] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:32.382759 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:32.384195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:32.392135 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:32.471388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfad15-212f-4028-8733-f7c3db0fd331 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:32.477129 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:32.477876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:32.478283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:32.478568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:32.501927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:32.503489 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:32.532407 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1362/2733] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:33.219919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:33.219919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:33.235931 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84b156-cae7-41d0-9f22-d60ff2f89898 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:33.247119 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:33.248541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:33.248980 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:33.249339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] PATH_INFO: `/v3/projects/d7fd3eafe3e246ffa5e2e4f6b0095307` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:33.252959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7fd3eafe3e246ffa5e2e4f6b0095307)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:33.255576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:33.325395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7fd3eafe3e246ffa5e2e4f6b0095307', 'request_id': 'req-82e0d833-5603-4ef5-92e2-13925f732409'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.344494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7fd3eafe3e246ffa5e2e4f6b0095307', 'request_id': 'req-82e0d833-5603-4ef5-92e2-13925f732409'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.352921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7fd3eafe3e246ffa5e2e4f6b0095307', 'request_id': 'req-82e0d833-5603-4ef5-92e2-13925f732409'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.366913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] The token cache is being invalidate because project d7fd3eafe3e246ffa5e2e4f6b0095307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:33.367342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.367938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.373796 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1372/2734] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:33 2022] DELETE /identity/v3/projects/d7fd3eafe3e246ffa5e2e4f6b0095307 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:33.375094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:33.385709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79925250-ce92-47c6-9d52-df7c8f4e73b2 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:33.461684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:33.462674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:33.462885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:33.463173 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] PATH_INFO: `/v3/users/85e0c3d974bf43cf9281cadbce79c7ca` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:33.464591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e0c3d974bf43cf9281cadbce79c7ca)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:33.466981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:33.534400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e0c3d974bf43cf9281cadbce79c7ca'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.545086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e0c3d974bf43cf9281cadbce79c7ca'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.559152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e0c3d974bf43cf9281cadbce79c7ca'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.559975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e0c3d974bf43cf9281cadbce79c7ca'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.560633 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e0c3d974bf43cf9281cadbce79c7ca'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:33.581813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1363/2735] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:33 2022] DELETE /identity/v3/users/85e0c3d974bf43cf9281cadbce79c7ca => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:33.863445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:33.865450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:33.873928 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:33.879683 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0d833-5603-4ef5-92e2-13925f732409 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:33.940442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:33.941339 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:33.941625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:33.941903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] PATH_INFO: `/v3/projects/4dbbe2860ee74ce39761e5ea7c0b0605` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:33.943705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dbbe2860ee74ce39761e5ea7c0b0605)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:33.946398 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:33.996690 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dbbe2860ee74ce39761e5ea7c0b0605', 'request_id': 'req-3fe419cb-ebf7-4206-b1f0-b555ad441a93'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.004544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dbbe2860ee74ce39761e5ea7c0b0605', 'request_id': 'req-3fe419cb-ebf7-4206-b1f0-b555ad441a93'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.010859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dbbe2860ee74ce39761e5ea7c0b0605', 'request_id': 'req-3fe419cb-ebf7-4206-b1f0-b555ad441a93'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.022597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] The token cache is being invalidate because project 4dbbe2860ee74ce39761e5ea7c0b0605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:34.022888 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.023550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-3fe419cb-ebf7-4206-b1f0-b555ad441a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.025136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1373/2736] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:33 2022] DELETE /identity/v3/projects/4dbbe2860ee74ce39761e5ea7c0b0605 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:34.029952 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:34.031568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:34.044339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cd659-ecd9-4917-8ed3-7edaed2fa47f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:34.064241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:34.102815 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:34.103687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:34.103968 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:34.104242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] PATH_INFO: `/v3/users/2b6eb1987bee4f65a05ead476b9c544b` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:34.105436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6eb1987bee4f65a05ead476b9c544b)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:34.107469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:34.158390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6eb1987bee4f65a05ead476b9c544b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.160242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78036eda-0429-42d5-89c8-1e26385971aa None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:34.160545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78036eda-0429-42d5-89c8-1e26385971aa None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:34.160867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78036eda-0429-42d5-89c8-1e26385971aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:34.167385 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6eb1987bee4f65a05ead476b9c544b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.174885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6eb1987bee4f65a05ead476b9c544b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.175811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6eb1987bee4f65a05ead476b9c544b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.176431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6eb1987bee4f65a05ead476b9c544b'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.180508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-78036eda-0429-42d5-89c8-1e26385971aa None None] MFA Rules not processed for user `066086bfb0f142978c8493b64e55dc15`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:34.181807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-78036eda-0429-42d5-89c8-1e26385971aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:34.194797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1364/2737] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:34 2022] DELETE /identity/v3/users/2b6eb1987bee4f65a05ead476b9c544b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:34.250166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1374/2738] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:34.541300 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:34.542938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:34.639466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a416854-16de-4c44-801a-f47e8809fd43 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:34.644237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:34.645152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:34.645437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:34.645715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] PATH_INFO: `/v3/projects/4e4644bcfa3048549766acecddf822e1` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:34.646957 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4644bcfa3048549766acecddf822e1)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:34.648950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:34.710465 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:34.714447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4644bcfa3048549766acecddf822e1', 'request_id': 'req-2d8c411f-0597-417a-b14c-6c7294003236'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.722902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4644bcfa3048549766acecddf822e1', 'request_id': 'req-2d8c411f-0597-417a-b14c-6c7294003236'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.729490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4644bcfa3048549766acecddf822e1', 'request_id': 'req-2d8c411f-0597-417a-b14c-6c7294003236'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.742872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] The token cache is being invalidate because project 4e4644bcfa3048549766acecddf822e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:34.743165 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.743925 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2d8c411f-0597-417a-b14c-6c7294003236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:34.745499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1365/2739] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:34 2022] DELETE /identity/v3/projects/4e4644bcfa3048549766acecddf822e1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:34.751581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c524148-7eb1-412b-b257-c1ba472bf11c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:34.752131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c524148-7eb1-412b-b257-c1ba472bf11c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:34.752131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c524148-7eb1-412b-b257-c1ba472bf11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:34.770512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6c524148-7eb1-412b-b257-c1ba472bf11c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:34.772380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c524148-7eb1-412b-b257-c1ba472bf11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:34.806655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:34.840542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1375/2740] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:34.849529 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b30d-b0d2-403c-8248-a5924da40282 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:34.849885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b30d-b0d2-403c-8248-a5924da40282 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:34.852416 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b30d-b0d2-403c-8248-a5924da40282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:34.878233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a254b30d-b0d2-403c-8248-a5924da40282 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:34.879606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a254b30d-b0d2-403c-8248-a5924da40282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:34.890698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1366/2741] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:34.895182 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c524148-7eb1-412b-b257-c1ba472bf11c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:34.907786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c524148-7eb1-412b-b257-c1ba472bf11c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:34.917310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:34.918078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:34.918681 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:34.919269 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:34.920474 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:56:34.920985 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:34.922737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:34.941192 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1376/2742] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:56:34.946412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b30d-b0d2-403c-8248-a5924da40282 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:34.958078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b30d-b0d2-403c-8248-a5924da40282 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:34.969328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:34.970156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:34.972992 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:34.973406 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:34.974929 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:34.977433 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.005921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1367/2743] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:34 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:35.011749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.022341 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dd98b-3d5e-409c-8adf-0fe71f1342ea admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.092434 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.093277 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.093571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.093853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.094544 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.096673 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.118198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1377/2744] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:35 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:35.122474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.133853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83db69b-c441-4445-9715-9cf12a3e200f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.139318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.140027 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.140312 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.140705 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.141386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.143156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.155502 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1368/2745] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:35.159862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.170519 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b5c7-cb8f-479f-945e-c4eae3cf6c49 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.175554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faa8f6-fe11-4b5d-9fe1-b2901f3a8c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.176218 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faa8f6-fe11-4b5d-9fe1-b2901f3a8c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.176505 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faa8f6-fe11-4b5d-9fe1-b2901f3a8c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.176811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faa8f6-fe11-4b5d-9fe1-b2901f3a8c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.177595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faa8f6-fe11-4b5d-9fe1-b2901f3a8c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.179197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faa8f6-fe11-4b5d-9fe1-b2901f3a8c8e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.189894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1378/2746] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:35.194226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.205186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fc919-53e6-476c-9046-f806cadec069 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.210818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.212052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.212420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.212776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] PATH_INFO: `/v3/projects/0b1d237c649342eeaf32c10ad1e59ddb/users/f08b264dbe264935abdbb59ea51e65bf/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.235428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1d237c649342eeaf32c10ad1e59ddb, user_id=f08b264dbe264935abdbb59ea51e65bf, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.238265 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.254326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1369/2747] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:35 2022] PUT /identity/v3/projects/0b1d237c649342eeaf32c10ad1e59ddb/users/f08b264dbe264935abdbb59ea51e65bf/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:35.321225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7957f-b24f-4ea5-a450-ed2798c494f8 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.321736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7957f-b24f-4ea5-a450-ed2798c494f8 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.322425 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7957f-b24f-4ea5-a450-ed2798c494f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.355469 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9ac7957f-b24f-4ea5-a450-ed2798c494f8 None None] MFA Rules not processed for user `f08b264dbe264935abdbb59ea51e65bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:35.356732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9ac7957f-b24f-4ea5-a450-ed2798c494f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:35.424524 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1379/2748] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:35.430217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.497208 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48991832-f51f-4c92-9956-7be08dcdd7d1 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.502021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.502676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.502955 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.503321 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.503910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.505868 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.539753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1370/2749] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:35 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:35.540894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7957f-b24f-4ea5-a450-ed2798c494f8 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.553092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7957f-b24f-4ea5-a450-ed2798c494f8 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.588547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:35.617442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.618118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.618407 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.618687 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.619395 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.621349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.643738 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1380/2750] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:35 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:56:35.645363 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.656564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20beb100-c432-4894-adc4-563108769a65 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.661140 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.661818 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.662103 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.662377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.663089 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.664709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.675106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1371/2751] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:35.679772 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.690253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4ce4e-c2e0-45d7-b57e-a73b69e3e5e0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.695782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.696759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.697050 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.697323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] PATH_INFO: `/v3/projects/5e694fe0ed9a41ce892508dd3917ce99/users/ca41e3323ace49498567898f01423590/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.714714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e694fe0ed9a41ce892508dd3917ce99, user_id=ca41e3323ace49498567898f01423590, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.717994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.733890 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1381/2752] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:35 2022] PUT /identity/v3/projects/5e694fe0ed9a41ce892508dd3917ce99/users/ca41e3323ace49498567898f01423590/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:35.738194 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.750168 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1ea56-337a-4562-9832-5d92b6efd3f9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.795676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:35.817695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58c56a-3750-4102-a2f4-235a84cd2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.818731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58c56a-3750-4102-a2f4-235a84cd2e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.819187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58c56a-3750-4102-a2f4-235a84cd2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.819756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58c56a-3750-4102-a2f4-235a84cd2e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.820932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58c56a-3750-4102-a2f4-235a84cd2e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.823630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58c56a-3750-4102-a2f4-235a84cd2e7b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.836698 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1372/2753] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:56:35.840791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:35.852933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe149a-a110-44b0-a97f-cbbf0b9ad034 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:35.859500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:35.861093 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.861385 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.861664 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] PATH_INFO: `/v3/projects/5e694fe0ed9a41ce892508dd3917ce99/users/ca41e3323ace49498567898f01423590/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.863842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e694fe0ed9a41ce892508dd3917ce99, user_id=ca41e3323ace49498567898f01423590, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:35.866666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:35.877925 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1382/2754] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:35 2022] PUT /identity/v3/projects/5e694fe0ed9a41ce892508dd3917ce99/users/ca41e3323ace49498567898f01423590/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:35.889769 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:35.890064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:35.890384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:35.928876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] MFA Rules not processed for user `ca41e3323ace49498567898f01423590`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:35.930099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.048115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1373/2755] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:36.052449 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:36.056892 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.062534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:36.064055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.126988 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91b58b-412c-4563-b29e-ff51ce2413c0 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:36.133103 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:36.133758 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:36.134044 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:36.134325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:36.137258 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.137365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69541634-965e-4400-9215-76dd9d8a5916 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:36.142733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:36.143397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:36.143683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:36.143963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:36.156700 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:36.158149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:36.170606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1374/2756] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:56:36.218054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:36.220249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:36.233716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1383/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:56:36.268108 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:36.277090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc33db-ce0c-428b-a445-19654c344bde service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:36.281021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:36.281697 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:36.281982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:36.282257 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:36.352189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:36.354687 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:36.368187 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1375/2758] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:36.692311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:36.704582 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc3cf8-359c-4b68-a4bc-9ec90f7c2bef service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:36.710619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:36.711462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:36.711753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:36.712032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] PATH_INFO: `/v3/users/f08b264dbe264935abdbb59ea51e65bf` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:36.713510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08b264dbe264935abdbb59ea51e65bf)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:36.716732 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:36.762378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:36.763874 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.776284 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=100 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:36.807404 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:36.834591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08b264dbe264935abdbb59ea51e65bf'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:36.845874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08b264dbe264935abdbb59ea51e65bf'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:36.859001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08b264dbe264935abdbb59ea51e65bf'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:36.859857 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08b264dbe264935abdbb59ea51e65bf'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:36.860774 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08b264dbe264935abdbb59ea51e65bf'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:36.877261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff19c8c-b641-404d-8a72-a9d39812e10d service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:36.880082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1384/2759] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:36 2022] DELETE /identity/v3/users/f08b264dbe264935abdbb59ea51e65bf => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:36.882255 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:36.884236 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.888458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6692c-5303-45eb-80d5-88d9b5c00b58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:36.889852 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6692c-5303-45eb-80d5-88d9b5c00b58 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:36.890149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6692c-5303-45eb-80d5-88d9b5c00b58 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:36.890430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6692c-5303-45eb-80d5-88d9b5c00b58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:36.892938 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b1d6692c-5303-45eb-80d5-88d9b5c00b58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.906866 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6692c-5303-45eb-80d5-88d9b5c00b58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:36.908476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6692c-5303-45eb-80d5-88d9b5c00b58 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:36.930887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1376/2760] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:56:36.934430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:36.935490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:36.935963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:36.959954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:36.961707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:36.969090 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9cf29-e2c1-4db9-9f22-dfd78df28dcd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:36.976260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:36.976898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:36.977056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1377/2761] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:36.977369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:36.977726 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] PATH_INFO: `/v3/users/8b60a3f963f34bbab49682c9d7a80bb5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:36.979127 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b60a3f963f34bbab49682c9d7a80bb5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:36.981200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:36.985203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:36.986317 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:37.044792 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b60a3f963f34bbab49682c9d7a80bb5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.055819 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b60a3f963f34bbab49682c9d7a80bb5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.065489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b60a3f963f34bbab49682c9d7a80bb5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.067038 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b60a3f963f34bbab49682c9d7a80bb5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.068575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b60a3f963f34bbab49682c9d7a80bb5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.072973 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841cbd2-66c7-44f7-9053-48468a8fdb5e None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:37.080707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:37.081581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:37.081895 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:37.082295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:37.085871 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:37.099696 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:37.099900 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1385/2762] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:36 2022] DELETE /identity/v3/users/8b60a3f963f34bbab49682c9d7a80bb5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:37.101344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:37.111855 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1378/2763] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:56:37.424349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:37.426180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:37.470025 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:37.507313 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53384-b46d-497d-88be-912b1dd8213d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:37.512135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:37.513034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:37.513315 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:37.513585 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] PATH_INFO: `/v3/projects/267db8931ade4a2d8658ccf487dad772` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:37.514782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=267db8931ade4a2d8658ccf487dad772)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:37.516751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:37.569021 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267db8931ade4a2d8658ccf487dad772', 'request_id': 'req-e11720d8-cf92-4745-b8a2-47420cf32853'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.578555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267db8931ade4a2d8658ccf487dad772', 'request_id': 'req-e11720d8-cf92-4745-b8a2-47420cf32853'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.592654 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267db8931ade4a2d8658ccf487dad772', 'request_id': 'req-e11720d8-cf92-4745-b8a2-47420cf32853'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.606574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] The token cache is being invalidate because project 267db8931ade4a2d8658ccf487dad772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:37.606790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.607453 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.609066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1386/2764] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:37 2022] DELETE /identity/v3/projects/267db8931ade4a2d8658ccf487dad772 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:37.659137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:37.661402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:37.710526 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f42079-b454-4c90-ad5a-70b1e19ab562 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:37.779205 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:37.780996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:37.816259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:37.819589 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:37.820102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:37.820457 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] PATH_INFO: `/v3/projects/0b1d237c649342eeaf32c10ad1e59ddb` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:37.822171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1d237c649342eeaf32c10ad1e59ddb)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:37.825563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:37.861516 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11720d8-cf92-4745-b8a2-47420cf32853 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:37.866446 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:37.867340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:37.867626 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:37.867905 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] PATH_INFO: `/v3/users/cf3828054d1647db99c8daf321c5c398` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:37.869174 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3828054d1647db99c8daf321c5c398)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:37.871108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:37.872856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:37.888341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1d237c649342eeaf32c10ad1e59ddb', 'request_id': 'req-0ad9847c-9d25-4e9d-919d-5020824d5977'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.899354 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1d237c649342eeaf32c10ad1e59ddb', 'request_id': 'req-0ad9847c-9d25-4e9d-919d-5020824d5977'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.907189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1d237c649342eeaf32c10ad1e59ddb', 'request_id': 'req-0ad9847c-9d25-4e9d-919d-5020824d5977'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.921281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] The token cache is being invalidate because project 0b1d237c649342eeaf32c10ad1e59ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:37.921628 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.922949 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.926587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1379/2765] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:37 2022] DELETE /identity/v3/projects/0b1d237c649342eeaf32c10ad1e59ddb => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:37.930492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:37.933645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:37.945430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3828054d1647db99c8daf321c5c398'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.955191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3828054d1647db99c8daf321c5c398'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.962364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:37.965703 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3828054d1647db99c8daf321c5c398'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.966489 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3828054d1647db99c8daf321c5c398'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.967204 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-41379b4d-73ed-4d61-87e6-1a801ef42405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3828054d1647db99c8daf321c5c398'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:37.991670 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1387/2766] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:37 2022] DELETE /identity/v3/users/cf3828054d1647db99c8daf321c5c398 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:37.995760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:37.996051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:37.996331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.011124 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9847c-9d25-4e9d-919d-5020824d5977 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:38.014911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:38.016139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:38.086776 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:38.087773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.090927 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.091043 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1388/2767] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:38.091122 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] PATH_INFO: `/v3/users/ca41e3323ace49498567898f01423590` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.093832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca41e3323ace49498567898f01423590)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:38.095785 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:38.097433 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:38.100106 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:38.153401 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca41e3323ace49498567898f01423590'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.163490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca41e3323ace49498567898f01423590'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.172078 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca41e3323ace49498567898f01423590'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.172838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca41e3323ace49498567898f01423590'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.173577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca41e3323ace49498567898f01423590'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.174291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242497-98f8-47a0-bc1a-127e1b457854 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:38.179149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:38.179909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.180283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.180614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.182100 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:38.182459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:38.195073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1380/2768] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:37 2022] DELETE /identity/v3/users/ca41e3323ace49498567898f01423590 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:38.195643 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:38.197125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:38.207387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1389/2769] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:56:38.608082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:38.609606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:38.621081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:38.622554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:38.646845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1496b-e19f-4e88-a2d5-e84fb6ed7231 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:38.685760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72153951-bfd0-4e47-9dea-e1ad0b467f4b admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:38.690614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:38.691521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.691854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.692142 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] PATH_INFO: `/v3/projects/5e694fe0ed9a41ce892508dd3917ce99` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.693381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e694fe0ed9a41ce892508dd3917ce99)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:38.695324 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:38.712765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:38.713629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.713904 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.714171 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] PATH_INFO: `/v3/projects/49df37ab58464b9dbb9d79d8a6ba5c23` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.715346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] RBAC: Authorizing `identity:delete_project(project_id=49df37ab58464b9dbb9d79d8a6ba5c23)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:38.717854 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:38.743213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:38.743943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e694fe0ed9a41ce892508dd3917ce99', 'request_id': 'req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.753210 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e694fe0ed9a41ce892508dd3917ce99', 'request_id': 'req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.759344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e694fe0ed9a41ce892508dd3917ce99', 'request_id': 'req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.769091 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49df37ab58464b9dbb9d79d8a6ba5c23', 'request_id': 'req-656a1e9c-ce64-4b52-a018-a20f9edcfeff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.770779 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] The token cache is being invalidate because project 5e694fe0ed9a41ce892508dd3917ce99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:38.771621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.772279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-dd39ba8d-a308-4351-ae6e-60d5d3ba190a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.774803 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1381/2770] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:38 2022] DELETE /identity/v3/projects/5e694fe0ed9a41ce892508dd3917ce99 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:38.777705 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49df37ab58464b9dbb9d79d8a6ba5c23', 'request_id': 'req-656a1e9c-ce64-4b52-a018-a20f9edcfeff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.782105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9aa6-33b7-4d85-a89c-1727efdc1b25 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.782422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9aa6-33b7-4d85-a89c-1727efdc1b25 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.782827 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9aa6-33b7-4d85-a89c-1727efdc1b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.785264 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49df37ab58464b9dbb9d79d8a6ba5c23', 'request_id': 'req-656a1e9c-ce64-4b52-a018-a20f9edcfeff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.797715 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] The token cache is being invalidate because project 49df37ab58464b9dbb9d79d8a6ba5c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:38.798003 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.798653 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-656a1e9c-ce64-4b52-a018-a20f9edcfeff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:38.800457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1390/2771] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:38 2022] DELETE /identity/v3/projects/49df37ab58464b9dbb9d79d8a6ba5c23 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:38.804809 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-342a9aa6-33b7-4d85-a89c-1727efdc1b25 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:38.806074 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-342a9aa6-33b7-4d85-a89c-1727efdc1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:38.873291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1382/2772] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:38.882382 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4953-4f58-4565-9519-1dbdbfa42d44 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.882895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4953-4f58-4565-9519-1dbdbfa42d44 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.882998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4953-4f58-4565-9519-1dbdbfa42d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.902909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3fda4953-4f58-4565-9519-1dbdbfa42d44 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:38.904098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3fda4953-4f58-4565-9519-1dbdbfa42d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:38.916309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1391/2773] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:38.918655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9aa6-33b7-4d85-a89c-1727efdc1b25 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:38.930959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9aa6-33b7-4d85-a89c-1727efdc1b25 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:38.941169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:38.941862 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.942141 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.942411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.943252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:56:38.943555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:38.945313 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:38.959672 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1383/2774] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:56:38.964129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4953-4f58-4565-9519-1dbdbfa42d44 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:38.973986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4953-4f58-4565-9519-1dbdbfa42d44 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:38.979763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:38.980478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:38.980798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:38.981074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:38.981794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:38.983747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:39.015018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1392/2775] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:38 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:56:39.016510 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:39.030280 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899756e-fb15-4784-8659-fa520611d3fa admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:39.034500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:39.095733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:39.096398 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:39.096709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:39.096985 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:39.097669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:39.100181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:39.119627 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1384/2776] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:39 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:39.123898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:39.134674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23c6e-4095-4040-8a5b-951b89bf1837 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:39.139217 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:39.139917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:39.140203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:39.140478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:39.141231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:39.142927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:39.153834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1393/2777] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:39.157656 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:39.167942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f149-ce09-4c83-b27c-7bb7970a1384 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:39.172486 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8341f3b-bd1e-4127-ab9e-3baa6a419719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:39.173179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8341f3b-bd1e-4127-ab9e-3baa6a419719 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:39.173464 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8341f3b-bd1e-4127-ab9e-3baa6a419719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:39.173736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8341f3b-bd1e-4127-ab9e-3baa6a419719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:39.174449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8341f3b-bd1e-4127-ab9e-3baa6a419719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:39.176279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8341f3b-bd1e-4127-ab9e-3baa6a419719 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:39.186296 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1385/2778] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:39.190487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:39.201057 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8d22e-4db9-4a4a-ba20-5386bdb1af95 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:39.205845 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:39.206782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:39.207062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:39.207336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] PATH_INFO: `/v3/projects/84c52238da1e4133a9ade64e9446830f/users/0f4a25f163b64a30bdf8a8defd5bad6d/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:39.222684 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c52238da1e4133a9ade64e9446830f, user_id=0f4a25f163b64a30bdf8a8defd5bad6d, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:39.225623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:39.238403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1394/2779] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:39 2022] PUT /identity/v3/projects/84c52238da1e4133a9ade64e9446830f/users/0f4a25f163b64a30bdf8a8defd5bad6d/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:56:39.248885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fd63-118a-467e-abd9-7789fad0612b None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:39.249174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fd63-118a-467e-abd9-7789fad0612b None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:39.249453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fd63-118a-467e-abd9-7789fad0612b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:39.276303 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-d8d2fd63-118a-467e-abd9-7789fad0612b None None] MFA Rules not processed for user `0f4a25f163b64a30bdf8a8defd5bad6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:39.277531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d8d2fd63-118a-467e-abd9-7789fad0612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:39.339889 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1386/2780] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:39.350536 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:39.352183 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:39.409105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0524f-a785-4e8f-951c-3e5063f4be8b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:39.412713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc09f5d-85aa-49b6-a0fe-ddb916e501b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:39.413376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc09f5d-85aa-49b6-a0fe-ddb916e501b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:39.413803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc09f5d-85aa-49b6-a0fe-ddb916e501b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:39.413936 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc09f5d-85aa-49b6-a0fe-ddb916e501b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:39.422649 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc09f5d-85aa-49b6-a0fe-ddb916e501b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:39.424065 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc09f5d-85aa-49b6-a0fe-ddb916e501b3 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:39.433099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1395/2781] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:42.490863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fd63-118a-467e-abd9-7789fad0612b None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:42.557836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fd63-118a-467e-abd9-7789fad0612b None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:42.562520 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:42.563386 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:42.563669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:42.563936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] PATH_INFO: `/v3/users/0f4a25f163b64a30bdf8a8defd5bad6d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:42.565130 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4a25f163b64a30bdf8a8defd5bad6d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:42.567032 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:42.585011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:42.620524 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4a25f163b64a30bdf8a8defd5bad6d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:42.629283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4a25f163b64a30bdf8a8defd5bad6d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:42.641389 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4a25f163b64a30bdf8a8defd5bad6d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:42.642166 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4a25f163b64a30bdf8a8defd5bad6d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:42.642790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4a25f163b64a30bdf8a8defd5bad6d'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:42.662695 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1387/2782] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:42 2022] DELETE /identity/v3/users/0f4a25f163b64a30bdf8a8defd5bad6d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:42.667431 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:42.668026 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:42.668120 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:42.686816 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:42.688162 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:42.735831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:42.752490 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1396/2783] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:42.762097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:42.763515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:42.821939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd50c0f-527f-4ded-8fae-0d4219497a4f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:42.825683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7675e1a-dd4a-4566-9362-04784c8eafa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:42.826328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7675e1a-dd4a-4566-9362-04784c8eafa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:42.826606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7675e1a-dd4a-4566-9362-04784c8eafa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:42.826885 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7675e1a-dd4a-4566-9362-04784c8eafa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:42.836458 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7675e1a-dd4a-4566-9362-04784c8eafa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:42.837910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7675e1a-dd4a-4566-9362-04784c8eafa9 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:42.847422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1388/2784] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:43.173697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.175543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:43.189105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734b3df-e7f8-48a6-84d4-87585031ed79 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.194764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.195665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.195948 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.196225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] PATH_INFO: `/v3/projects/84c52238da1e4133a9ade64e9446830f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.197704 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c52238da1e4133a9ade64e9446830f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.199893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.275110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c52238da1e4133a9ade64e9446830f', 'request_id': 'req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:43.284842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c52238da1e4133a9ade64e9446830f', 'request_id': 'req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:43.291785 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c52238da1e4133a9ade64e9446830f', 'request_id': 'req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:43.304542 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] The token cache is being invalidate because project 84c52238da1e4133a9ade64e9446830f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:43.305230 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:43.305691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4beccd27-9f7a-443b-85ef-d6ed976ab8ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:43.307640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1397/2785] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:43 2022] DELETE /identity/v3/projects/84c52238da1e4133a9ade64e9446830f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:43.312781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec38c-18da-4ced-b3a5-cbebebde8412 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.313347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec38c-18da-4ced-b3a5-cbebebde8412 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.313701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fec38c-18da-4ced-b3a5-cbebebde8412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.332681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-85fec38c-18da-4ced-b3a5-cbebebde8412 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:43.334428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-85fec38c-18da-4ced-b3a5-cbebebde8412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:43.399221 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1389/2786] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:56:43.407089 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930c4a0-53c5-4643-ab5b-296487679552 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.407478 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930c4a0-53c5-4643-ab5b-296487679552 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.407764 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930c4a0-53c5-4643-ab5b-296487679552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.426834 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b930c4a0-53c5-4643-ab5b-296487679552 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:43.428728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b930c4a0-53c5-4643-ab5b-296487679552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:43.437156 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1398/2787] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:43.440979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fec38c-18da-4ced-b3a5-cbebebde8412 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.451709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fec38c-18da-4ced-b3a5-cbebebde8412 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.456371 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.457063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.457347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.457623 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.458361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:56:43.458673 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.460261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.471707 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1390/2788] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:56:43.476420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930c4a0-53c5-4643-ab5b-296487679552 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.485751 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930c4a0-53c5-4643-ab5b-296487679552 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.491118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.491709 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.491995 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.492303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.493040 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.494939 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.520650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1399/2789] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:43 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:43.524805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.535251 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051a2b2-93e8-4929-99ca-628fa1f4a209 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.557390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:43.595875 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.596544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.596863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.597143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.597837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.599802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.620087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1391/2790] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:43 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:56:43.624178 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.635350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562ad4c-40f6-4b35-bee6-50c1502b4c32 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.639987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.640676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.640964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.641243 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.641969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.643555 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.653829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1400/2791] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:43.657780 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.668281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6649b-703e-4926-8130-223ed0097c92 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.672918 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af625491-4ccf-4140-a5fc-eff33afc60df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.673568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af625491-4ccf-4140-a5fc-eff33afc60df admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.673861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af625491-4ccf-4140-a5fc-eff33afc60df admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.674117 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af625491-4ccf-4140-a5fc-eff33afc60df admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.674838 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af625491-4ccf-4140-a5fc-eff33afc60df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.676690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af625491-4ccf-4140-a5fc-eff33afc60df admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.687181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1392/2792] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:43.691229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.701523 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b595567-53f1-48ab-80c9-355b49eedc75 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.706210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.707081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.707390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.707668 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] PATH_INFO: `/v3/projects/3812900f5aca4fbf884304a2dd0353ee/users/4297896b40e749b19798fe84ad3b7074/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.722073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3812900f5aca4fbf884304a2dd0353ee, user_id=4297896b40e749b19798fe84ad3b7074, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.725232 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.737693 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1401/2793] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:43 2022] PUT /identity/v3/projects/3812900f5aca4fbf884304a2dd0353ee/users/4297896b40e749b19798fe84ad3b7074/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:43.749226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40d179-8783-435f-853f-021e9da49fb4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.749456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40d179-8783-435f-853f-021e9da49fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.749735 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40d179-8783-435f-853f-021e9da49fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.777339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5d40d179-8783-435f-853f-021e9da49fb4 None None] MFA Rules not processed for user `4297896b40e749b19798fe84ad3b7074`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:43.778579 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5d40d179-8783-435f-853f-021e9da49fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:43.842430 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1393/2794] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:43.854115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:43.855859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:43.912730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8226445-1958-4460-8d41-0c3c47889a75 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:43.916390 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d6162-e23b-412d-a335-5a5c0eb53cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:43.917079 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d6162-e23b-412d-a335-5a5c0eb53cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:43.917367 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d6162-e23b-412d-a335-5a5c0eb53cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:43.917652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d6162-e23b-412d-a335-5a5c0eb53cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:43.926309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d6162-e23b-412d-a335-5a5c0eb53cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:43.928350 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d6162-e23b-412d-a335-5a5c0eb53cb3 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:43.937247 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1402/2795] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:46.468154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40d179-8783-435f-853f-021e9da49fb4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:46.531519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40d179-8783-435f-853f-021e9da49fb4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:46.536090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:46.537609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:46.537894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:46.538175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] PATH_INFO: `/v3/users/4297896b40e749b19798fe84ad3b7074` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:46.539369 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4297896b40e749b19798fe84ad3b7074)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:46.541423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:46.584223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:46.594864 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4297896b40e749b19798fe84ad3b7074'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:46.602652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4297896b40e749b19798fe84ad3b7074'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:46.609797 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4297896b40e749b19798fe84ad3b7074'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:46.610435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4297896b40e749b19798fe84ad3b7074'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:46.611038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4297896b40e749b19798fe84ad3b7074'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:46.629178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1394/2796] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:56:46 2022] DELETE /identity/v3/users/4297896b40e749b19798fe84ad3b7074 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:46.635370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:46.635869 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:46.635963 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:46.653328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:46.654528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:46.659737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:46.721267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1403/2797] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:46.731023 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:46.732447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:46.796035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ae24e-c618-4386-a58a-178868b229d4 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:46.799676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d7c02-cc89-4e6b-8c28-b564a2db6c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:46.800620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7c02-cc89-4e6b-8c28-b564a2db6c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:46.800907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7c02-cc89-4e6b-8c28-b564a2db6c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:46.801186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7c02-cc89-4e6b-8c28-b564a2db6c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:46.811554 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d7c02-cc89-4e6b-8c28-b564a2db6c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:46.813038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d7c02-cc89-4e6b-8c28-b564a2db6c15 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:46.822557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1395/2798] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:47.102729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.104139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:47.116109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c06469-6549-408d-970a-3dbea7d40423 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.121032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.121672 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.121954 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.122228 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] PATH_INFO: `/v3/projects/3812900f5aca4fbf884304a2dd0353ee` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.123467 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3812900f5aca4fbf884304a2dd0353ee)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.125408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.178663 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3812900f5aca4fbf884304a2dd0353ee', 'request_id': 'req-bb3e4641-f528-4a0a-8b75-dc6861cba008'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:47.186242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3812900f5aca4fbf884304a2dd0353ee', 'request_id': 'req-bb3e4641-f528-4a0a-8b75-dc6861cba008'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:47.191884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3812900f5aca4fbf884304a2dd0353ee', 'request_id': 'req-bb3e4641-f528-4a0a-8b75-dc6861cba008'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:47.202908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] The token cache is being invalidate because project 3812900f5aca4fbf884304a2dd0353ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:56:47.203200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:47.203861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-bb3e4641-f528-4a0a-8b75-dc6861cba008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:56:47.205490 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1404/2799] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:56:47 2022] DELETE /identity/v3/projects/3812900f5aca4fbf884304a2dd0353ee => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:47.211614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde16f8-6e15-4344-9945-7e13abecead4 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.211894 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde16f8-6e15-4344-9945-7e13abecead4 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.212181 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde16f8-6e15-4344-9945-7e13abecead4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.230392 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-0dde16f8-6e15-4344-9945-7e13abecead4 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:47.231622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-0dde16f8-6e15-4344-9945-7e13abecead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:47.290629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1396/2800] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:47.298762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af948-6964-4c10-81de-0acb2d36ef0c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.299048 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af948-6964-4c10-81de-0acb2d36ef0c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.299321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af948-6964-4c10-81de-0acb2d36ef0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.321311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-eb3af948-6964-4c10-81de-0acb2d36ef0c None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:47.322520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-eb3af948-6964-4c10-81de-0acb2d36ef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:47.333189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1405/2801] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:47.339009 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde16f8-6e15-4344-9945-7e13abecead4 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.350744 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde16f8-6e15-4344-9945-7e13abecead4 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.355492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.356228 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.356519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.356850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.357587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:56:47.357900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.359563 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.375970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1397/2802] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 09:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:56:47.377191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3af948-6964-4c10-81de-0acb2d36ef0c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.387527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3af948-6964-4c10-81de-0acb2d36ef0c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.393140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.393882 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.394238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.394612 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.395318 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.397336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.429635 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1406/2803] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 09:56:47 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:47.430865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.441639 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d4902-3e40-408e-b64d-05041adea13a admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.485029 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:47.504710 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.505477 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.505799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.506083 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.506781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.508747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.529134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1398/2804] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 09:56:47 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:56:47.533399 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.544249 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8c9ed-4f73-4968-a66f-df5c017a0d2d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.548849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.549503 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.549782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.550056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.551039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.553316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.564799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1407/2805] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:47.569017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.579292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ad892-d126-40e4-8b69-5d9e2cf2f0e9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.584071 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd2a1b-c5e4-402d-8589-72351eedf865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.584773 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2a1b-c5e4-402d-8589-72351eedf865 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.585062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2a1b-c5e4-402d-8589-72351eedf865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.585343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2a1b-c5e4-402d-8589-72351eedf865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.586073 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd2a1b-c5e4-402d-8589-72351eedf865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.587663 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd2a1b-c5e4-402d-8589-72351eedf865 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.597736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1399/2806] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 09:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:56:47.602810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.613580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11071db6-b545-40b5-9959-3da66ad5ea5d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.619233 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.620128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.620411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.620714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] PATH_INFO: `/v3/projects/f0bde152fd404d8ea15548019bcfe9f5/users/548b5aca3de042b6b1e093f9f39f8c24/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.635810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0bde152fd404d8ea15548019bcfe9f5, user_id=548b5aca3de042b6b1e093f9f39f8c24, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.638655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.653116 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1408/2807] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 09:56:47 2022] PUT /identity/v3/projects/f0bde152fd404d8ea15548019bcfe9f5/users/548b5aca3de042b6b1e093f9f39f8c24/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:56:47.663884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3666993-fd73-4954-a30d-46846e1a82bb None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.664178 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3666993-fd73-4954-a30d-46846e1a82bb None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.664461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3666993-fd73-4954-a30d-46846e1a82bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.692227 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a3666993-fd73-4954-a30d-46846e1a82bb None None] MFA Rules not processed for user `548b5aca3de042b6b1e093f9f39f8c24`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:56:47.693478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a3666993-fd73-4954-a30d-46846e1a82bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:47.779499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1400/2808] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:56:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:56:47.791832 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:56:47.793554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:56:47.838271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:47.868113 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dc552-5ede-403e-9eed-5903704a7330 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:56:47.872966 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:56:47.873632 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:56:47.873921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:56:47.874210 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:56:47.883913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:56:47.885411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:56:47.895655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1409/2809] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:56:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:56:57.775712 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:57.890811 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=12 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:57:00.680481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1b815-2644-4588-addd-65da1edc358d None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:00.680996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1b815-2644-4588-addd-65da1edc358d None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:00.681086 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1b815-2644-4588-addd-65da1edc358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:00.745305 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-68c1b815-2644-4588-addd-65da1edc358d None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:57:00.746515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-68c1b815-2644-4588-addd-65da1edc358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:00.811772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1401/2810] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Jun 24 09:57:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:57:00.857859 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:00.859423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:00.953595 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa6df8-c692-4cb4-a76f-2e63b661d2a7 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:00.959035 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10aa71-b632-48e7-bca2-5088612e4f05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:00.959685 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10aa71-b632-48e7-bca2-5088612e4f05 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:00.959975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10aa71-b632-48e7-bca2-5088612e4f05 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:00.960256 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10aa71-b632-48e7-bca2-5088612e4f05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:00.968521 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10aa71-b632-48e7-bca2-5088612e4f05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:00.970149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10aa71-b632-48e7-bca2-5088612e4f05 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:00.979115 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1410/2811] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:57:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:57:01.123924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1b815-2644-4588-addd-65da1edc358d None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:01.204984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1b815-2644-4588-addd-65da1edc358d None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:01.210020 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:01.211055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:01.211414 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:01.211791 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] PATH_INFO: `/v3/users/548b5aca3de042b6b1e093f9f39f8c24` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:01.213150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=548b5aca3de042b6b1e093f9f39f8c24)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:01.215158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:01.272580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548b5aca3de042b6b1e093f9f39f8c24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:01.283030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548b5aca3de042b6b1e093f9f39f8c24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:01.301625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548b5aca3de042b6b1e093f9f39f8c24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:01.303116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548b5aca3de042b6b1e093f9f39f8c24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:01.304016 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548b5aca3de042b6b1e093f9f39f8c24'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:01.307292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:57:01.335782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1402/2812] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:57:01 2022] DELETE /identity/v3/users/548b5aca3de042b6b1e093f9f39f8c24 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:57:01.340511 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:01.340846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:01.341132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:01.365459 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:57:01.366762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:01.451734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1411/2813] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 09:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:57:01.462281 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:01.463719 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:01.530708 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156415-af16-46a6-887d-6e1e07b99d9d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:01.534979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07aea0-acf0-4204-a0eb-831a494ac895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:01.535637 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07aea0-acf0-4204-a0eb-831a494ac895 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:01.535951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07aea0-acf0-4204-a0eb-831a494ac895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:01.536224 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07aea0-acf0-4204-a0eb-831a494ac895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:01.546496 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07aea0-acf0-4204-a0eb-831a494ac895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:01.547954 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07aea0-acf0-4204-a0eb-831a494ac895 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:01.559996 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1403/2814] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:57:01.889697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:01.891714 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:01.905104 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b09a6-873c-4ec1-b31b-00287f530519 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:01.912518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:01.913921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:01.914271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:01.914856 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] PATH_INFO: `/v3/projects/f0bde152fd404d8ea15548019bcfe9f5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:01.916644 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0bde152fd404d8ea15548019bcfe9f5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:01.919809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:01.995488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0bde152fd404d8ea15548019bcfe9f5', 'request_id': 'req-4c2346b9-8bea-4e05-a813-911be20978e2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:02.009328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0bde152fd404d8ea15548019bcfe9f5', 'request_id': 'req-4c2346b9-8bea-4e05-a813-911be20978e2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:02.016884 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0bde152fd404d8ea15548019bcfe9f5', 'request_id': 'req-4c2346b9-8bea-4e05-a813-911be20978e2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:02.018167 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:57:02.045742 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] The token cache is being invalidate because project f0bde152fd404d8ea15548019bcfe9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:57:02.046036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:02.046707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:02.048603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1412/2815] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:57:01 2022] DELETE /identity/v3/projects/f0bde152fd404d8ea15548019bcfe9f5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:57:11.555476 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=32,keystone:DELETE=2 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:57:12.042279 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:57:16.178279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:16.178749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:16.178847 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:16.196340 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] MFA Rules not processed for user `b2370d988bdf4499a1a3e71e8336d002`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:57:16.197580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:16.253139 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1404/2816] 10.222.0.52 () {58 vars in 967 bytes} [Fri Jun 24 09:57:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:57:16.265748 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:16.267164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:16.337512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2346b9-8bea-4e05-a813-911be20978e2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:16.342360 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbaf3f85739348beb1bc8bae579f0da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:16.343005 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:16.343289 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:16.343562 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:16.351763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:16.353359 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:16.361309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1413/2817] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Jun 24 09:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:57:18.262022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:18.263523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:18.332347 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe324d-b74a-4380-b040-4b84a6a29f75 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:18.337190 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:18.338066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:18.338350 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:18.338625 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] PATH_INFO: `/v3/users/066086bfb0f142978c8493b64e55dc15` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:18.339828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=066086bfb0f142978c8493b64e55dc15)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:18.341796 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:18.389614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066086bfb0f142978c8493b64e55dc15'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.396766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066086bfb0f142978c8493b64e55dc15'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.404662 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066086bfb0f142978c8493b64e55dc15'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.405461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066086bfb0f142978c8493b64e55dc15'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.406186 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066086bfb0f142978c8493b64e55dc15'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.424087 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1405/2818] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 09:57:18 2022] DELETE /identity/v3/users/066086bfb0f142978c8493b64e55dc15 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:57:18.600952 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:18.617739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936fd7a-7b43-438e-9b38-25dff61df220 service swift] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:18.680674 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:18.681531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:18.681814 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:18.682092 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] PATH_INFO: `/v3/projects/3d7cd74c906c4307ae9a7b36a763ba3d` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:18.683283 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7cd74c906c4307ae9a7b36a763ba3d)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:18.685224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:18.731370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7cd74c906c4307ae9a7b36a763ba3d', 'request_id': 'req-fc67b84d-73e9-4158-a390-4155750c60ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.738636 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7cd74c906c4307ae9a7b36a763ba3d', 'request_id': 'req-fc67b84d-73e9-4158-a390-4155750c60ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.744245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7cd74c906c4307ae9a7b36a763ba3d', 'request_id': 'req-fc67b84d-73e9-4158-a390-4155750c60ed'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.755138 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] The token cache is being invalidate because project 3d7cd74c906c4307ae9a7b36a763ba3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:57:18.755420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.756049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:57:18.757727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1414/2819] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 09:57:18 2022] DELETE /identity/v3/projects/3d7cd74c906c4307ae9a7b36a763ba3d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:57:28.420534 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:57:28.752041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:57:44.661114 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:44.662547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:44.719740 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408f42-1f0e-44f2-b7b0-dadc1ca57c03 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:44.767979 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:44.768724 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:44.769007 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:44.769285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:44.771682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:44.842240 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:44.843592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:44.855335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1406/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:57:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:57:52.705081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:57:52.706498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:52.719615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b84d-73e9-4158-a390-4155750c60ed admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:57:52.723145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:57:52.723781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:57:52.724060 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:57:52.724336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:57:52.726131 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:57:52.739883 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:57:52.741327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:57:52.752227 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1415/2821] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:57:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:57:54.850372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=74 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:58:02.747340 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:58:28.395169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:58:28.396578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:58:28.485260 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7256e7-8745-42d8-95df-202ee17118a6 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:58:28.491236 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:58:28.491882 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:58:28.492179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:58:28.492450 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:58:28.494322 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:58:28.507110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:58:28.509821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:58:28.524152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1407/2822] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 09:58:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:58:38.519788 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=37 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:00:08.556736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:00:08.565749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1b7f4-23b5-4e78-bb71-dc0234b1f682 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:00:08.569605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:00:08.570238 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:00:08.570518 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:00:08.570790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:00:08.572550 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:00:08.643865 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:00:08.645281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:00:08.664302 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1416/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:00:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:00:18.661662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=36 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:03:35.427239 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:03:35.428679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:03:35.450781 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaa6ae-541d-4d63-a8b0-1a1c08e89f7a service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:03:35.455631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:03:35.456254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:03:35.456535 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:03:35.456836 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:03:35.466163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:03:35.467739 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:03:35.477670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1408/2824] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:03:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:03:45.473425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=10 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:01.345370 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:01.346791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:01.403345 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442e2c2-8cc1-443d-ac86-2cae236b66c1 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:01.406920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:01.407556 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:01.407836 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:01.408108 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:01.410198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:01.495655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:01.497132 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:01.517311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1417/2825] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:04.500318 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.510636 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff757c7-2254-48ad-a118-5fb2c2e28352 service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.515642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.516278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.516645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.516946 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.518728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:04.530308 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.531869 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.542807 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1409/2826] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:04.561284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.562619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:04.575197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61121f-92c0-4181-a01c-bdc2583ab2bc service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.580410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.581327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.581627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.582082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] PATH_INFO: `/v3/users/f8333f6a9dab4618bc0e6b2060f15250` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.583722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8333f6a9dab4618bc0e6b2060f15250)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.585716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.633723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8333f6a9dab4618bc0e6b2060f15250'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.640579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8333f6a9dab4618bc0e6b2060f15250'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.647494 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8333f6a9dab4618bc0e6b2060f15250'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.648123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8333f6a9dab4618bc0e6b2060f15250'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.648815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-292a81f1-506c-4b9a-92af-65b89e605b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8333f6a9dab4618bc0e6b2060f15250'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.658793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:04.668974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1418/2827] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:04 2022] DELETE /identity/v3/users/f8333f6a9dab4618bc0e6b2060f15250 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:04.841479 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:04.855593 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4d3cb-9fee-44de-9e61-fe4a1d04d00c service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:04.913096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:04.913960 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.914247 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.914528 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] PATH_INFO: `/v3/projects/67221e1503c34e3d9fa3f13cd40c37dc` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:04.915701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67221e1503c34e3d9fa3f13cd40c37dc)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:04.917657 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:04.962237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67221e1503c34e3d9fa3f13cd40c37dc', 'request_id': 'req-80d57bdd-489f-4af6-b200-5f330a806630'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.968900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67221e1503c34e3d9fa3f13cd40c37dc', 'request_id': 'req-80d57bdd-489f-4af6-b200-5f330a806630'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.975066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67221e1503c34e3d9fa3f13cd40c37dc', 'request_id': 'req-80d57bdd-489f-4af6-b200-5f330a806630'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.986158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] The token cache is being invalidate because project 67221e1503c34e3d9fa3f13cd40c37dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:04.986440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.987065 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-80d57bdd-489f-4af6-b200-5f330a806630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:04.988659 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1410/2828] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:04 2022] DELETE /identity/v3/projects/67221e1503c34e3d9fa3f13cd40c37dc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:04.994416 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daa07b-946e-4878-a152-4b2a8d6fd574 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:04.994711 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daa07b-946e-4878-a152-4b2a8d6fd574 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:04.994993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daa07b-946e-4878-a152-4b2a8d6fd574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.023242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-81daa07b-946e-4878-a152-4b2a8d6fd574 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:05.024475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-81daa07b-946e-4878-a152-4b2a8d6fd574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.083881 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1419/2829] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:05.091504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21183-ff05-43c7-97e2-02c3589cd546 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.091760 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21183-ff05-43c7-97e2-02c3589cd546 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.092036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21183-ff05-43c7-97e2-02c3589cd546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.109880 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-10b21183-ff05-43c7-97e2-02c3589cd546 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:05.111701 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-10b21183-ff05-43c7-97e2-02c3589cd546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.120067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1411/2830] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:05.124122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daa07b-946e-4878-a152-4b2a8d6fd574 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.134285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daa07b-946e-4878-a152-4b2a8d6fd574 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.138901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.139554 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.139842 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.140118 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.140878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:05.141244 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.142862 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.154849 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1420/2831] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:05.158746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b21183-ff05-43c7-97e2-02c3589cd546 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.167856 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b21183-ff05-43c7-97e2-02c3589cd546 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.172426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.173171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.173452 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.173725 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.174679 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.176550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.201490 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1412/2832] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:05 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:05.205820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.216158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aecdde-d23b-49d5-9ebf-5608ab829b59 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.271376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.272030 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.272311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.272608 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.273282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.275180 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.294867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1421/2833] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:05 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:05.298872 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.309388 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2de85-12af-42f8-abd6-11253a290163 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.314000 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.314653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.314987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.315264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.315986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.318112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.328295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1413/2834] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:05.332581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.342455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551cc67-439f-40b2-8e80-3fb9357c2ec2 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.346997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1d742-820c-4488-b9c5-289c0083a661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.347645 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d742-820c-4488-b9c5-289c0083a661 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.347926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d742-820c-4488-b9c5-289c0083a661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.348203 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1d742-820c-4488-b9c5-289c0083a661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.348965 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1d742-820c-4488-b9c5-289c0083a661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.350530 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1d742-820c-4488-b9c5-289c0083a661 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.360224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1422/2835] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:05.363953 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.373711 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a663fe-4059-40f4-80d8-e70f4b05cf9f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.378431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.379805 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.380091 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.380370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] PATH_INFO: `/v3/projects/8471c9a73bd04e9d98127d4e627983ce/users/ba90ee0c8f9340f5bc16aef493238c99/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.395372 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8471c9a73bd04e9d98127d4e627983ce, user_id=ba90ee0c8f9340f5bc16aef493238c99, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.398036 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.410963 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1414/2836] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:05 2022] PUT /identity/v3/projects/8471c9a73bd04e9d98127d4e627983ce/users/ba90ee0c8f9340f5bc16aef493238c99/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:05.421323 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.421613 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.421893 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.448348 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] MFA Rules not processed for user `ba90ee0c8f9340f5bc16aef493238c99`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:05.449597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.489325 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.514800 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1423/2837] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:04:05.526975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.530557 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.546025 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:05.608220 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed55c7-e19d-4656-9fff-83e134bb4753 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.614564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.615391 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.615684 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.615958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.623934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.625570 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.635297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1415/2838] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:05.716745 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.718122 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.773667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3572d-f951-49fc-8bb7-ad390baf6468 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.777272 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.777910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.778195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.778468 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.780179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.793676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.795085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.806728 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1424/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:05.871984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:05.873409 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.885249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b38e-470a-458e-8b97-30f710f70fbd service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:05.888784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de716d70-023d-40a3-837c-fbd5630ca1db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:05.889413 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de716d70-023d-40a3-837c-fbd5630ca1db service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:05.889693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de716d70-023d-40a3-837c-fbd5630ca1db service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:05.889966 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de716d70-023d-40a3-837c-fbd5630ca1db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:05.891650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-de716d70-023d-40a3-837c-fbd5630ca1db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:05.904511 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de716d70-023d-40a3-837c-fbd5630ca1db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:05.905959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de716d70-023d-40a3-837c-fbd5630ca1db service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:05.917201 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1416/2840] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:05.991098 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.053418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef4b4d-0950-4e27-89b7-4ca8a85acfc4 service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.057983 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.058848 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.059133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.059411 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] PATH_INFO: `/v3/users/ba90ee0c8f9340f5bc16aef493238c99` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.060619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba90ee0c8f9340f5bc16aef493238c99)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.062622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.107799 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba90ee0c8f9340f5bc16aef493238c99'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.114838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba90ee0c8f9340f5bc16aef493238c99'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.121739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba90ee0c8f9340f5bc16aef493238c99'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.122320 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba90ee0c8f9340f5bc16aef493238c99'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.122938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba90ee0c8f9340f5bc16aef493238c99'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.141655 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1425/2841] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:05 2022] DELETE /identity/v3/users/ba90ee0c8f9340f5bc16aef493238c99 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:06.146772 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.147060 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.147337 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.163765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:06.164989 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:06.223888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1417/2842] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:06.233895 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.235321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:06.241309 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:06.294590 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e17f31-3fec-4474-84e7-8c9fff2af8a1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.298266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82921d37-9e4b-4af0-b864-58a6205e369a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.298909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82921d37-9e4b-4af0-b864-58a6205e369a service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.299196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82921d37-9e4b-4af0-b864-58a6205e369a service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.299734 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82921d37-9e4b-4af0-b864-58a6205e369a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.309209 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82921d37-9e4b-4af0-b864-58a6205e369a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.310692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82921d37-9e4b-4af0-b864-58a6205e369a service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.321109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1426/2843] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:06.577887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.579377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:06.594472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93c4e8-f28a-4fd7-9185-ee8de3ea7ce0 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.599896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.600794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.601082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.601362 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] PATH_INFO: `/v3/projects/8471c9a73bd04e9d98127d4e627983ce` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.602565 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8471c9a73bd04e9d98127d4e627983ce)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.605548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.653767 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8471c9a73bd04e9d98127d4e627983ce', 'request_id': 'req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.661498 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8471c9a73bd04e9d98127d4e627983ce', 'request_id': 'req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.666890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8471c9a73bd04e9d98127d4e627983ce', 'request_id': 'req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.678487 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] The token cache is being invalidate because project 8471c9a73bd04e9d98127d4e627983ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:06.678778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.679440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-00cb9cc0-8e4e-4f6f-8d75-ad56523fc23e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:06.681002 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1418/2844] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:06 2022] DELETE /identity/v3/projects/8471c9a73bd04e9d98127d4e627983ce => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:06.685806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7a3b6-98ff-404b-9714-a137b2ead3dd None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.686102 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7a3b6-98ff-404b-9714-a137b2ead3dd None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.686386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7a3b6-98ff-404b-9714-a137b2ead3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.704502 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9ff7a3b6-98ff-404b-9714-a137b2ead3dd None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:06.705730 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9ff7a3b6-98ff-404b-9714-a137b2ead3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:06.764747 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1427/2845] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:06.772326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edfff2-5cd6-437f-87ef-fa104278d346 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.772665 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edfff2-5cd6-437f-87ef-fa104278d346 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.772944 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edfff2-5cd6-437f-87ef-fa104278d346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.791211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-36edfff2-5cd6-437f-87ef-fa104278d346 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:06.792383 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-36edfff2-5cd6-437f-87ef-fa104278d346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:06.800366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1419/2846] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:06.804853 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7a3b6-98ff-404b-9714-a137b2ead3dd None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.815144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7a3b6-98ff-404b-9714-a137b2ead3dd None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.821088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.821749 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.822037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.822310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.823034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:06.823334 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.824942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.836492 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1428/2847] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:06.840778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edfff2-5cd6-437f-87ef-fa104278d346 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.845112 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:06.852050 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edfff2-5cd6-437f-87ef-fa104278d346 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.858374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.859054 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.859343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.859621 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.860335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.862343 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.891282 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1420/2848] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:06 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:04:06.896133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:06.906543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9c4cb-6ac1-4892-a448-3e67364e015e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:06.967145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:06.967731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:06.968018 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:06.968299 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:06.969039 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:06.970959 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:06.992019 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1429/2849] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:06 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:04:06.996275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.007154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1412ca8-c433-4bdc-b9be-f5ea2045ce32 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.011670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.012315 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.012630 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.012907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.013624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.015199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.025956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1421/2850] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:07.029912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.040081 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbe259-0cf2-4396-be93-33c3ae44917f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.044846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cb84c-834b-4518-b70e-fa64fe6013a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.045506 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cb84c-834b-4518-b70e-fa64fe6013a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.045790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cb84c-834b-4518-b70e-fa64fe6013a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.046095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cb84c-834b-4518-b70e-fa64fe6013a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.046820 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cb84c-834b-4518-b70e-fa64fe6013a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.048401 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cb84c-834b-4518-b70e-fa64fe6013a5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.059117 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1430/2851] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:07.063102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.073488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af550b1f-0e67-4d41-9624-a32bdeed9498 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.078287 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.079162 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.079556 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.079790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] PATH_INFO: `/v3/projects/c6bfdc6b36ea4c1e87b09e9ff324cef3/users/95ee1f0e776a4cd5b96689d7935a90b4/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.094794 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6bfdc6b36ea4c1e87b09e9ff324cef3, user_id=95ee1f0e776a4cd5b96689d7935a90b4, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.097513 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.110030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1422/2852] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:07 2022] PUT /identity/v3/projects/c6bfdc6b36ea4c1e87b09e9ff324cef3/users/95ee1f0e776a4cd5b96689d7935a90b4/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:07.120803 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2098c-7db4-423c-bc13-01126d365520 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.121094 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2098c-7db4-423c-bc13-01126d365520 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.121374 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2098c-7db4-423c-bc13-01126d365520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.140790 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:07.148798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e1d2098c-7db4-423c-bc13-01126d365520 None None] MFA Rules not processed for user `95ee1f0e776a4cd5b96689d7935a90b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:07.150166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e1d2098c-7db4-423c-bc13-01126d365520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:07.213847 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1431/2853] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:07.218341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.279649 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46df43-bee2-4881-b539-ee53fc2cc9c1 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.284193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.284910 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.285189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.285463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.286138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.288005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.312799 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1423/2854] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:07 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:04:07.317409 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2098c-7db4-423c-bc13-01126d365520 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.328074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2098c-7db4-423c-bc13-01126d365520 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.382599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.383257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.383597 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.383873 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.384577 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.386481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.406634 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1432/2855] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:07 2022] POST /identity/v3/users => generated 309 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:07.411151 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.421189 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765857c-8f94-45de-9a29-fa40ac55a3da admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.430204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.431319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.431765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.432259 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.433241 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.435058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.446116 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1424/2856] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:07.450328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.460297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861740f-ef97-4273-8cd4-23a9b959efe9 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.464982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.465851 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.466139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.466447 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] PATH_INFO: `/v3/projects/e55446e6927b4e9e810c2f1feb3ca61f/users/35d29383206543e392a9bcd388cd030a/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.481510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55446e6927b4e9e810c2f1feb3ca61f, user_id=35d29383206543e392a9bcd388cd030a, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.484133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.497783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1433/2857] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:07 2022] PUT /identity/v3/projects/e55446e6927b4e9e810c2f1feb3ca61f/users/35d29383206543e392a9bcd388cd030a/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:07.501790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.511644 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda32b7-e72c-45ba-b52b-f4d69038a835 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.568606 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb2af2-2a1b-408c-a1c9-48df47e2379a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.569267 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2af2-2a1b-408c-a1c9-48df47e2379a admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.569555 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2af2-2a1b-408c-a1c9-48df47e2379a admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.569833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2af2-2a1b-408c-a1c9-48df47e2379a admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.570549 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2af2-2a1b-408c-a1c9-48df47e2379a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.572134 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2af2-2a1b-408c-a1c9-48df47e2379a admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.581821 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1425/2858] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:07.585887 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.596036 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d1688-8754-4bb4-9d22-d9e1febeb8d6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.601553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.602415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.602707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.602986 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] PATH_INFO: `/v3/projects/e55446e6927b4e9e810c2f1feb3ca61f/users/35d29383206543e392a9bcd388cd030a/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.604969 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55446e6927b4e9e810c2f1feb3ca61f, user_id=35d29383206543e392a9bcd388cd030a, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.607592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.618387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1434/2859] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:07 2022] PUT /identity/v3/projects/e55446e6927b4e9e810c2f1feb3ca61f/users/35d29383206543e392a9bcd388cd030a/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:07.632693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826738e-8f29-477a-8743-e69329d1d736 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.633067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826738e-8f29-477a-8743-e69329d1d736 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.633415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826738e-8f29-477a-8743-e69329d1d736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.667378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-b826738e-8f29-477a-8743-e69329d1d736 None None] MFA Rules not processed for user `35d29383206543e392a9bcd388cd030a`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:07.668602 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-b826738e-8f29-477a-8743-e69329d1d736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:07.677146 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:07.743542 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1426/2860] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:07 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:07.753987 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:07.755696 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:07.812281 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11d636-33c8-46dd-b6b3-b839b4ada399 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:07.815944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef81b3-d157-4ced-97ac-74e34f9078eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:07.816594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef81b3-d157-4ced-97ac-74e34f9078eb service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:07.816878 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef81b3-d157-4ced-97ac-74e34f9078eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:07.817151 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef81b3-d157-4ced-97ac-74e34f9078eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:07.823868 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:07.826198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef81b3-d157-4ced-97ac-74e34f9078eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:07.827909 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef81b3-d157-4ced-97ac-74e34f9078eb service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:07.838843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1435/2861] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:07 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:07.968143 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826738e-8f29-477a-8743-e69329d1d736 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.037299 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826738e-8f29-477a-8743-e69329d1d736 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:08.041850 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:08.042720 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.043001 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.043273 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] PATH_INFO: `/v3/users/95ee1f0e776a4cd5b96689d7935a90b4` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.044443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ee1f0e776a4cd5b96689d7935a90b4)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:08.046381 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.092726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ee1f0e776a4cd5b96689d7935a90b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.099912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ee1f0e776a4cd5b96689d7935a90b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.106721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ee1f0e776a4cd5b96689d7935a90b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.107379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ee1f0e776a4cd5b96689d7935a90b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.107975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ee1f0e776a4cd5b96689d7935a90b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.124967 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1427/2862] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:07 2022] DELETE /identity/v3/users/95ee1f0e776a4cd5b96689d7935a90b4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:08.129994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.130280 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.130553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.148224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:08.149475 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:08.206326 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1436/2863] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:08.216681 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.218067 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:08.239399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:08.278132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d97fb9-2c69-45fa-aba2-734bf151689f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:08.283553 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:08.284195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.284481 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.284896 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.296676 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:08.298055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.307840 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1428/2864] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:08.576929 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.579760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:08.602908 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104046-5ba6-4152-a5c2-8a27452c296f None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:08.610552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:08.611700 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.612031 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.612352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] PATH_INFO: `/v3/projects/c6bfdc6b36ea4c1e87b09e9ff324cef3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.614191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6bfdc6b36ea4c1e87b09e9ff324cef3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:08.616078 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.663142 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6bfdc6b36ea4c1e87b09e9ff324cef3', 'request_id': 'req-59aac506-4e56-4d42-93e7-9ffab1a9b971'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.670004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6bfdc6b36ea4c1e87b09e9ff324cef3', 'request_id': 'req-59aac506-4e56-4d42-93e7-9ffab1a9b971'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.675616 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6bfdc6b36ea4c1e87b09e9ff324cef3', 'request_id': 'req-59aac506-4e56-4d42-93e7-9ffab1a9b971'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.686622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] The token cache is being invalidate because project c6bfdc6b36ea4c1e87b09e9ff324cef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:08.686912 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.687970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.689591 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1437/2865] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:08 2022] DELETE /identity/v3/projects/c6bfdc6b36ea4c1e87b09e9ff324cef3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:08.693591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:08.703268 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330c398-62c9-4cb4-b110-0bc38de82676 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:08.757983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:08.758843 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:08.759123 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:08.759397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] PATH_INFO: `/v3/users/35d29383206543e392a9bcd388cd030a` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:08.760574 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d29383206543e392a9bcd388cd030a)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:08.762472 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:08.806860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d29383206543e392a9bcd388cd030a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.813380 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d29383206543e392a9bcd388cd030a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.820844 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d29383206543e392a9bcd388cd030a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.821408 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d29383206543e392a9bcd388cd030a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.822006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-2e37a60b-ba08-4c1f-bdc2-7ab3a88f96cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d29383206543e392a9bcd388cd030a'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:08.839196 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1429/2866] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:08 2022] DELETE /identity/v3/users/35d29383206543e392a9bcd388cd030a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:09.100169 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.102008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:09.115697 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aac506-4e56-4d42-93e7-9ffab1a9b971 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.171761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.172636 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.172916 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.173189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] PATH_INFO: `/v3/projects/e55446e6927b4e9e810c2f1feb3ca61f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.174611 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55446e6927b4e9e810c2f1feb3ca61f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.176543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.199176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.224692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55446e6927b4e9e810c2f1feb3ca61f', 'request_id': 'req-741323c8-0040-4702-ac86-731b16d47925'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:09.231886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55446e6927b4e9e810c2f1feb3ca61f', 'request_id': 'req-741323c8-0040-4702-ac86-731b16d47925'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:09.237324 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55446e6927b4e9e810c2f1feb3ca61f', 'request_id': 'req-741323c8-0040-4702-ac86-731b16d47925'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:09.248531 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] The token cache is being invalidate because project e55446e6927b4e9e810c2f1feb3ca61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:09.248914 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:09.249551 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-741323c8-0040-4702-ac86-731b16d47925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:09.251077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1438/2867] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:09 2022] DELETE /identity/v3/projects/e55446e6927b4e9e810c2f1feb3ca61f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:09.256734 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d8910-08ee-46a2-874c-8bb357e0b199 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.257022 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d8910-08ee-46a2-874c-8bb357e0b199 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.257306 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d8910-08ee-46a2-874c-8bb357e0b199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.274270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-667d8910-08ee-46a2-874c-8bb357e0b199 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:09.275453 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-667d8910-08ee-46a2-874c-8bb357e0b199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:09.313423 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:09.335694 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1430/2868] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:09.343471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c492-e843-4277-a59b-28fce353d0b5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.343762 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c492-e843-4277-a59b-28fce353d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.344037 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c492-e843-4277-a59b-28fce353d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.361823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b530c492-e843-4277-a59b-28fce353d0b5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:09.363528 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b530c492-e843-4277-a59b-28fce353d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:09.371342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1439/2869] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:09.375368 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d8910-08ee-46a2-874c-8bb357e0b199 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.385377 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d8910-08ee-46a2-874c-8bb357e0b199 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.389943 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.390591 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.390873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.391153 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.391873 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80156) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:09.392180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.393811 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.405131 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1431/2870] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:09.409121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530c492-e843-4277-a59b-28fce353d0b5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.418149 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530c492-e843-4277-a59b-28fce353d0b5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.422755 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.423386 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.423665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.423957 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.424919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.427291 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.452920 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1440/2871] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:09.456384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.466200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28497b-a099-47cb-940a-6121a5905c20 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.521531 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.522183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.522463 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.522736 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.523410 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.525330 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.543394 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1432/2872] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/users => generated 285 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:09.547342 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.558110 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da998d-4865-4b94-af8e-4e34e9d1bdc1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.562646 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.563296 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.563576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.563850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.564584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.566166 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.577846 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1441/2873] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:09.581536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.592163 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ccae-773e-4ca7-b26d-41e6e39d8708 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.596747 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663a483-f10e-4a0d-9d40-bf92e06e86e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.597397 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663a483-f10e-4a0d-9d40-bf92e06e86e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.597706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663a483-f10e-4a0d-9d40-bf92e06e86e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.598488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663a483-f10e-4a0d-9d40-bf92e06e86e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.599203 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663a483-f10e-4a0d-9d40-bf92e06e86e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.600829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663a483-f10e-4a0d-9d40-bf92e06e86e4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.610690 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1433/2874] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:09.615301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.624996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535330d6-b117-4b09-92f8-4d8a3a9703e6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.629580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.630441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.630725 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.630998 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] PATH_INFO: `/v3/projects/6a7f6f9cc8ab44e3acc8b0d886154989/users/35efe14cca4842e1a7e26994427df5e5/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.645015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7f6f9cc8ab44e3acc8b0d886154989, user_id=35efe14cca4842e1a7e26994427df5e5, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.647628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.660795 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1442/2875] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:09 2022] PUT /identity/v3/projects/6a7f6f9cc8ab44e3acc8b0d886154989/users/35efe14cca4842e1a7e26994427df5e5/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:09.665645 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04aff8-ecae-40ed-863b-c74ad322dfeb None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.665956 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04aff8-ecae-40ed-863b-c74ad322dfeb None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.666237 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04aff8-ecae-40ed-863b-c74ad322dfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.684314 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-1a04aff8-ecae-40ed-863b-c74ad322dfeb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:09.685523 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-1a04aff8-ecae-40ed-863b-c74ad322dfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:09.743264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1434/2876] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:04:09.752647 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:09.754347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:09.810810 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fb667-4557-4bab-8740-a840df6911f8 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:09.814427 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:09.815062 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:09.815351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:09.815629 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:09.824460 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:09.825898 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:09.835173 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1443/2877] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:10.423198 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b33fe1-c3d4-40cb-9daa-011fba06f814 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.423497 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b33fe1-c3d4-40cb-9daa-011fba06f814 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.423778 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b33fe1-c3d4-40cb-9daa-011fba06f814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.451107 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-37b33fe1-c3d4-40cb-9daa-011fba06f814 None None] MFA Rules not processed for user `35efe14cca4842e1a7e26994427df5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:10.452326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-37b33fe1-c3d4-40cb-9daa-011fba06f814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:10.492064 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.520478 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1435/2878] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:10.524956 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.534831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b95f-0334-459e-bc24-67d13ad6902a service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.539423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.540067 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.540352 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.540650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.541343 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.543257 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.570087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1444/2879] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:10 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:10.573845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b33fe1-c3d4-40cb-9daa-011fba06f814 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.585366 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b33fe1-c3d4-40cb-9daa-011fba06f814 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.640297 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.640921 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.641232 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.641514 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] PATH_INFO: `/v3/users` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.642242 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.644156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.667185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1436/2880] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:10 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:10.670944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.681977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ac6e-61bf-457d-9764-5cb14b9623ae admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.686481 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.687130 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.687413 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.687691 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.688418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.690105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.701071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1445/2881] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:10.705185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.715581 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05046e53-26db-4a44-9065-086be230b881 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.720195 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.721115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.721400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.721678 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] PATH_INFO: `/v3/projects/434412a8483847f89715686613388b7c/users/4f2d359537364880aeeafab986257391/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.736411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434412a8483847f89715686613388b7c, user_id=4f2d359537364880aeeafab986257391, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.739642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.752344 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1437/2882] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:10 2022] PUT /identity/v3/projects/434412a8483847f89715686613388b7c/users/4f2d359537364880aeeafab986257391/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:04:10.756880 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.766798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23db205-d43b-498c-804c-51d1c0403d8d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.777919 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:10.824152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064503f-f1fb-4a29-9c37-8920da05817d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.824850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064503f-f1fb-4a29-9c37-8920da05817d admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.825136 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064503f-f1fb-4a29-9c37-8920da05817d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.825417 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064503f-f1fb-4a29-9c37-8920da05817d admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.826140 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064503f-f1fb-4a29-9c37-8920da05817d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.827722 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064503f-f1fb-4a29-9c37-8920da05817d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.837392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1446/2883] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:10.841402 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:10.851480 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bbf6bf-7f7b-4ed2-b102-6c9d74750269 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:10.856092 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:10.857033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:10.857316 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:10.857596 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] PATH_INFO: `/v3/projects/434412a8483847f89715686613388b7c/users/4f2d359537364880aeeafab986257391/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:10.859543 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=434412a8483847f89715686613388b7c, user_id=4f2d359537364880aeeafab986257391, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:10.862185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:10.872616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1438/2884] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:10 2022] PUT /identity/v3/projects/434412a8483847f89715686613388b7c/users/4f2d359537364880aeeafab986257391/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:11.497332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.497737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.498015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.525526 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] MFA Rules not processed for user `4f2d359537364880aeeafab986257391`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:11.526731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:11.597605 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1447/2885] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:11.609011 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:11.610765 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:11.667536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b0090-7620-4878-b3e9-d5f739cdbd3c admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:11.671035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:11.671677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:11.671959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:11.672233 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:11.716429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:11.730096 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:11.731525 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:11.740370 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1439/2886] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:13.073331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:13.074746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:13.086924 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d911d6-2c55-4ccc-a5de-c683a70e8875 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:13.090541 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:13.091168 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:13.091451 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:13.091723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:13.101152 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:13.102563 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:13.112054 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1448/2887] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:21.736346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=7 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:23.107614 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=62 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:25.325801 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:25.327234 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:25.399721 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3cd9d-4838-464a-8280-8905abb318f6 service cinder] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:25.404870 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:25.405540 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:25.405824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:25.406101 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:25.407945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:25.421404 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:25.423057 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:25.435536 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1440/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:26.120455 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.182794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de21f3-2d1e-4903-b8cb-4b61975a06ec service cinder] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:26.187262 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:26.188126 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.188410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.188727 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] PATH_INFO: `/v3/users/35efe14cca4842e1a7e26994427df5e5` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.189906 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35efe14cca4842e1a7e26994427df5e5)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:26.192316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:26.238123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35efe14cca4842e1a7e26994427df5e5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.244783 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35efe14cca4842e1a7e26994427df5e5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.251791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35efe14cca4842e1a7e26994427df5e5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.252527 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35efe14cca4842e1a7e26994427df5e5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.253161 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35efe14cca4842e1a7e26994427df5e5'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.271656 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1449/2889] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:26 2022] DELETE /identity/v3/users/35efe14cca4842e1a7e26994427df5e5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:26.443865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.455147 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c98bb-436f-470e-9076-ccf7ae3f7a50 service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:26.512440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:26.513336 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.513614 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.513884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] PATH_INFO: `/v3/projects/6a7f6f9cc8ab44e3acc8b0d886154989` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.515035 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7f6f9cc8ab44e3acc8b0d886154989)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:26.516993 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:26.560859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7f6f9cc8ab44e3acc8b0d886154989', 'request_id': 'req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.570262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7f6f9cc8ab44e3acc8b0d886154989', 'request_id': 'req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.575828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7f6f9cc8ab44e3acc8b0d886154989', 'request_id': 'req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.586500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] The token cache is being invalidate because project 6a7f6f9cc8ab44e3acc8b0d886154989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:26.586784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.587420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.588975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1441/2890] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:26 2022] DELETE /identity/v3/projects/6a7f6f9cc8ab44e3acc8b0d886154989 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:26.593007 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.594387 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:26.606951 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d57fb4-3101-4149-9d07-1592aea84301 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:26.661500 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:26.662351 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:26.662627 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:26.662894 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] PATH_INFO: `/v3/users/4f2d359537364880aeeafab986257391` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:26.664099 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2d359537364880aeeafab986257391)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:26.666032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:26.688083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:26.714692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2d359537364880aeeafab986257391'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.721514 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2d359537364880aeeafab986257391'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.728335 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2d359537364880aeeafab986257391'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.728996 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2d359537364880aeeafab986257391'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.729628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-5230b0ad-9212-4882-a43f-183024545b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2d359537364880aeeafab986257391'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:26.747088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1450/2891] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:26 2022] DELETE /identity/v3/users/4f2d359537364880aeeafab986257391 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:26.918345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:26.919865 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:26.947700 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a6f8-e00b-4154-ae4f-f5b8499a577f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:26.951518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.005831 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.006756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.007049 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.007326 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] PATH_INFO: `/v3/projects/434412a8483847f89715686613388b7c` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.008499 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=434412a8483847f89715686613388b7c)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.010424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.057958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434412a8483847f89715686613388b7c', 'request_id': 'req-82a0e704-ad76-462d-9018-566ba5e40232'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:27.064817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434412a8483847f89715686613388b7c', 'request_id': 'req-82a0e704-ad76-462d-9018-566ba5e40232'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:27.070447 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434412a8483847f89715686613388b7c', 'request_id': 'req-82a0e704-ad76-462d-9018-566ba5e40232'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:27.081790 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] The token cache is being invalidate because project 434412a8483847f89715686613388b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:27.082079 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:27.082728 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82a0e704-ad76-462d-9018-566ba5e40232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:27.084295 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1442/2892] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:26 2022] DELETE /identity/v3/projects/434412a8483847f89715686613388b7c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:27.089964 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c776e-2329-44b4-9e63-e472e5d4248a None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.090285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c776e-2329-44b4-9e63-e472e5d4248a None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.090566 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c776e-2329-44b4-9e63-e472e5d4248a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.108733 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-cc5c776e-2329-44b4-9e63-e472e5d4248a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:27.109933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-cc5c776e-2329-44b4-9e63-e472e5d4248a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:27.168129 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1451/2893] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:27.176863 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668e75f-f2ee-4d52-bc5e-1b0ce7261fe2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.177150 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668e75f-f2ee-4d52-bc5e-1b0ce7261fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.177428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668e75f-f2ee-4d52-bc5e-1b0ce7261fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.194748 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a668e75f-f2ee-4d52-bc5e-1b0ce7261fe2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:27.195892 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a668e75f-f2ee-4d52-bc5e-1b0ce7261fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:27.203654 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1443/2894] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:27.208271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c776e-2329-44b4-9e63-e472e5d4248a None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.217984 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c776e-2329-44b4-9e63-e472e5d4248a None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.223139 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.223788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.224071 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.224346 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.225105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:27.225410 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.226997 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.238363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1452/2895] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:27.242569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668e75f-f2ee-4d52-bc5e-1b0ce7261fe2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.251675 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668e75f-f2ee-4d52-bc5e-1b0ce7261fe2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.256304 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.256974 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.257252 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.257527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.258207 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.260185 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.295338 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1444/2896] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:27 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:27.299454 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.312782 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21219f3-9c59-41dc-aca5-0b23286104fd admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.367176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.367826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.368109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.368419 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.369133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.371024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.390135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1453/2897] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:27 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:27.394726 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.407716 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec10e4-bb63-44a9-960a-e45025be8783 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.413097 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.413741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.414021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.414293 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.415012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.416610 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.426597 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1445/2898] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:27.430557 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.440187 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20248df-a0ff-4373-941e-870548b5fb07 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.444874 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324d37c-ae15-40af-983a-5a72f043546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.445457 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324d37c-ae15-40af-983a-5a72f043546f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.445737 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324d37c-ae15-40af-983a-5a72f043546f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.446010 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324d37c-ae15-40af-983a-5a72f043546f admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.446723 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324d37c-ae15-40af-983a-5a72f043546f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.448278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324d37c-ae15-40af-983a-5a72f043546f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.453418 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.460017 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1454/2899] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:27.463689 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.474428 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c587c2-3832-4865-9820-20b46b3549fa admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.479109 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.479981 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.480272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.480580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] PATH_INFO: `/v3/projects/d8ab7ffc6a26428795dbd0164694ef91/users/71cb049cf91e45a883104c69cc3cba5f/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.494932 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ab7ffc6a26428795dbd0164694ef91, user_id=71cb049cf91e45a883104c69cc3cba5f, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.497642 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.510571 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1446/2900] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:27 2022] PUT /identity/v3/projects/d8ab7ffc6a26428795dbd0164694ef91/users/71cb049cf91e45a883104c69cc3cba5f/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:27.522077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15984-4845-45ca-84b3-b359617daf59 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.522364 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15984-4845-45ca-84b3-b359617daf59 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.522640 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15984-4845-45ca-84b3-b359617daf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.548807 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-e5b15984-4845-45ca-84b3-b359617daf59 None None] MFA Rules not processed for user `71cb049cf91e45a883104c69cc3cba5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:27.550266 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-e5b15984-4845-45ca-84b3-b359617daf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:27.613096 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1455/2901] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:27.624069 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:27.625786 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:27.649718 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:27.700521 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd07b-72b6-4034-9432-dffbcf91889b admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:27.704149 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8baabf9-714e-49fc-8ad7-a5b22c005aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:27.704820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baabf9-714e-49fc-8ad7-a5b22c005aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:27.705105 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baabf9-714e-49fc-8ad7-a5b22c005aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:27.705384 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baabf9-714e-49fc-8ad7-a5b22c005aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:27.714950 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8baabf9-714e-49fc-8ad7-a5b22c005aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:27.716378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8baabf9-714e-49fc-8ad7-a5b22c005aa8 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:27.725072 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1447/2902] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:37.610158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=32 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:37.721110 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=24 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:51.306128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b15984-4845-45ca-84b3-b359617daf59 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:51.387133 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b15984-4845-45ca-84b3-b359617daf59 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:51.391650 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:51.392517 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:51.392829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:51.393107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] PATH_INFO: `/v3/users/71cb049cf91e45a883104c69cc3cba5f` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:51.394786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71cb049cf91e45a883104c69cc3cba5f)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:51.396760 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:51.442739 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71cb049cf91e45a883104c69cc3cba5f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:51.450488 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71cb049cf91e45a883104c69cc3cba5f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:51.459231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71cb049cf91e45a883104c69cc3cba5f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:51.459877 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71cb049cf91e45a883104c69cc3cba5f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:51.460483 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71cb049cf91e45a883104c69cc3cba5f'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:51.478311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1456/2903] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:51 2022] DELETE /identity/v3/users/71cb049cf91e45a883104c69cc3cba5f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:51.482900 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:51.483379 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:51.483473 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:51.511017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:51.512199 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:51.569766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1448/2904] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:04:51.579329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:51.580786 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:51.638231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926870d7-ced8-4aee-b980-0316c939a260 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:51.641843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a12d8-b412-4545-8375-b5c69ea40425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:51.642482 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a12d8-b412-4545-8375-b5c69ea40425 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:51.642766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a12d8-b412-4545-8375-b5c69ea40425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:51.643041 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a12d8-b412-4545-8375-b5c69ea40425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:51.652579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a12d8-b412-4545-8375-b5c69ea40425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:51.654001 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a12d8-b412-4545-8375-b5c69ea40425 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:51.663285 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1457/2905] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:51.923817 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:51.925784 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:51.937887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bdf03-7f5e-4efc-8b47-a4c11ffcd9a2 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:51.943204 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:51.944066 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:51.944364 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:51.944670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] PATH_INFO: `/v3/projects/d8ab7ffc6a26428795dbd0164694ef91` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:51.945853 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ab7ffc6a26428795dbd0164694ef91)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:51.947742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:51.999837 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ab7ffc6a26428795dbd0164694ef91', 'request_id': 'req-93f154ac-6640-462b-b739-3eee96379207'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.007688 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ab7ffc6a26428795dbd0164694ef91', 'request_id': 'req-93f154ac-6640-462b-b739-3eee96379207'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.013415 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ab7ffc6a26428795dbd0164694ef91', 'request_id': 'req-93f154ac-6640-462b-b739-3eee96379207'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.024264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] The token cache is being invalidate because project d8ab7ffc6a26428795dbd0164694ef91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:52.024547 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.025219 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-93f154ac-6640-462b-b739-3eee96379207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.026746 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1449/2906] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:51 2022] DELETE /identity/v3/projects/d8ab7ffc6a26428795dbd0164694ef91 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:52.031525 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6aa5-7304-456d-9526-cd3ad7501e62 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.031815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6aa5-7304-456d-9526-cd3ad7501e62 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.032095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6aa5-7304-456d-9526-cd3ad7501e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.054940 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:52.071945 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-9cdf6aa5-7304-456d-9526-cd3ad7501e62 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:52.073158 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9cdf6aa5-7304-456d-9526-cd3ad7501e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:52.132665 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1458/2907] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:52.140062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3493-e6a9-4558-9acc-4edeaf3a70c6 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.140357 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3493-e6a9-4558-9acc-4edeaf3a70c6 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.140671 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3493-e6a9-4558-9acc-4edeaf3a70c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.158564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-ac9a3493-e6a9-4558-9acc-4edeaf3a70c6 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:52.159755 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-ac9a3493-e6a9-4558-9acc-4edeaf3a70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:52.167580 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1450/2908] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:52.171539 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf6aa5-7304-456d-9526-cd3ad7501e62 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.181872 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf6aa5-7304-456d-9526-cd3ad7501e62 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.186442 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.187107 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.187391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.187666 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.188392 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:52.188736 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.190310 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.202375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1459/2909] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 10:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:52.206474 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3493-e6a9-4558-9acc-4edeaf3a70c6 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.215501 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3493-e6a9-4558-9acc-4edeaf3a70c6 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.220082 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.220766 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.221052 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.221328 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.222019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.223970 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.261879 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1451/2910] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:52 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:04:52.263788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.273515 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b29c9-d55c-478f-bcf3-caf327ff275b admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.328231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.328923 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.329190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.329465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.330270 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.332198 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.351721 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1460/2911] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:52 2022] POST /identity/v3/users => generated 284 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:52.355677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.366714 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1104b-61af-4f0d-a908-5f843c1cfd1d admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.371342 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.371999 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.372292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.372607 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.373332 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.374945 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.385028 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1452/2912] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:52.388815 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.398552 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780d8d2-39f2-4430-a03a-65bfe5b753a7 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.403121 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932066e-6af3-49f4-86a1-f0dc149b2fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.403766 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932066e-6af3-49f4-86a1-f0dc149b2fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.404051 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932066e-6af3-49f4-86a1-f0dc149b2fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.404331 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932066e-6af3-49f4-86a1-f0dc149b2fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.405082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932066e-6af3-49f4-86a1-f0dc149b2fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.406652 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932066e-6af3-49f4-86a1-f0dc149b2fa3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.416886 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1461/2913] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:52.420733 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.431412 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa12b-cb27-4ea7-b760-4c1191343578 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.435975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.436876 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.437154 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.437426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] PATH_INFO: `/v3/projects/3c205f26cfc3429fb8e732625b1458cf/users/7cc69c302d6248969c5053237d196ab2/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.451802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c205f26cfc3429fb8e732625b1458cf, user_id=7cc69c302d6248969c5053237d196ab2, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.454492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.466703 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1453/2914] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:52 2022] PUT /identity/v3/projects/3c205f26cfc3429fb8e732625b1458cf/users/7cc69c302d6248969c5053237d196ab2/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:52.478128 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e2cce-5c00-4d2a-accb-c2e09fed65c5 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.478415 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e2cce-5c00-4d2a-accb-c2e09fed65c5 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.478695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e2cce-5c00-4d2a-accb-c2e09fed65c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.505765 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-908e2cce-5c00-4d2a-accb-c2e09fed65c5 None None] MFA Rules not processed for user `7cc69c302d6248969c5053237d196ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:52.506975 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-908e2cce-5c00-4d2a-accb-c2e09fed65c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:52.564066 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:52.574145 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1462/2915] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:52.585335 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.587010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:52.591670 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:52.666278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8aac59-aabe-4749-aaf2-8fce500060e9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.669890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa31a68-06c1-4067-b537-9a3c15fe31c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.670548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31a68-06c1-4067-b537-9a3c15fe31c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.670820 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31a68-06c1-4067-b537-9a3c15fe31c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.671099 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31a68-06c1-4067-b537-9a3c15fe31c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.680249 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa31a68-06c1-4067-b537-9a3c15fe31c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.681709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa31a68-06c1-4067-b537-9a3c15fe31c0 service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.690400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1454/2916] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:52.714206 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e2cce-5c00-4d2a-accb-c2e09fed65c5 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.774615 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e2cce-5c00-4d2a-accb-c2e09fed65c5 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:52.779123 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:52.779982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.780271 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.780592 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] PATH_INFO: `/v3/users/7cc69c302d6248969c5053237d196ab2` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.781789 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc69c302d6248969c5053237d196ab2)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:52.783699 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:52.828047 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc69c302d6248969c5053237d196ab2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.837109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc69c302d6248969c5053237d196ab2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.844831 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc69c302d6248969c5053237d196ab2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.845465 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc69c302d6248969c5053237d196ab2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.846083 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc69c302d6248969c5053237d196ab2'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:52.863901 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1463/2917] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:04:52 2022] DELETE /identity/v3/users/7cc69c302d6248969c5053237d196ab2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:52.868545 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:52.868860 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:52.869137 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:52.885655 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:52.886858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:52.945017 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1455/2918] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:52.954286 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:52.955692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:53.017581 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d13e4b-970d-45e0-8452-56d8c0c26e9d admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.021861 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6bb28-59fc-476f-ace0-328567ee6026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.022540 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6bb28-59fc-476f-ace0-328567ee6026 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.022858 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6bb28-59fc-476f-ace0-328567ee6026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.023135 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6bb28-59fc-476f-ace0-328567ee6026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.032470 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6bb28-59fc-476f-ace0-328567ee6026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.033927 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6bb28-59fc-476f-ace0-328567ee6026 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.043993 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1464/2919] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:04:53.301975 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.303396 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:53.316010 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9468e-7711-4624-99fc-1c28817e7aa5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.320620 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.321538 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.321816 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.322088 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] PATH_INFO: `/v3/projects/3c205f26cfc3429fb8e732625b1458cf` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.323346 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c205f26cfc3429fb8e732625b1458cf)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.325435 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.371015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c205f26cfc3429fb8e732625b1458cf', 'request_id': 'req-db41f58e-7f97-4006-b562-7b210aff77b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:53.378424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c205f26cfc3429fb8e732625b1458cf', 'request_id': 'req-db41f58e-7f97-4006-b562-7b210aff77b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:53.383789 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c205f26cfc3429fb8e732625b1458cf', 'request_id': 'req-db41f58e-7f97-4006-b562-7b210aff77b4'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:53.395058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] The token cache is being invalidate because project 3c205f26cfc3429fb8e732625b1458cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:04:53.395365 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:53.395983 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-db41f58e-7f97-4006-b562-7b210aff77b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:04:53.397544 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1456/2920] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:04:53 2022] DELETE /identity/v3/projects/3c205f26cfc3429fb8e732625b1458cf => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:53.402347 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ee52-a6b9-4450-8ebe-09cde52381c2 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.402822 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ee52-a6b9-4450-8ebe-09cde52381c2 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.402910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ee52-a6b9-4450-8ebe-09cde52381c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.419841 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-b195ee52-a6b9-4450-8ebe-09cde52381c2 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:53.421056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-b195ee52-a6b9-4450-8ebe-09cde52381c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:53.433498 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:53.481784 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1465/2921] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:53.489757 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c3c8-f18c-483a-89a1-d863f2f1465a None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.490047 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c3c8-f18c-483a-89a1-d863f2f1465a None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.490323 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c3c8-f18c-483a-89a1-d863f2f1465a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.508418 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-a202c3c8-f18c-483a-89a1-d863f2f1465a None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:53.509624 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-a202c3c8-f18c-483a-89a1-d863f2f1465a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:53.517285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1457/2922] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:53.521275 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195ee52-a6b9-4450-8ebe-09cde52381c2 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.530974 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195ee52-a6b9-4450-8ebe-09cde52381c2 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.535491 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.536137 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.536423 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.536759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.537512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:04:53.537823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.539414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.551778 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1466/2923] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:04:53.555527 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202c3c8-f18c-483a-89a1-d863f2f1465a None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.561019 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:04:53.564951 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202c3c8-f18c-483a-89a1-d863f2f1465a None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.571243 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.572095 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.572578 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.572914 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.573743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.575658 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.600546 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1458/2924] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:04:53 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:04:53.604794 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.615179 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc7a56-0aa0-4e84-bff2-62d0ae84e9c5 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.676314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.677015 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.677301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.677579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.678259 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.680199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.700046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1467/2925] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:04:53 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:04:53.704183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.714275 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafebe95-feb3-46f2-a595-fa4425d9dded admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.718859 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.719504 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.719787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.720063 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.720822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.722434 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.733263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1459/2926] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:04:53.736207 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.746809 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343b683-a838-4dc5-8709-095314cb7060 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.751311 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cac322-ff5c-445a-a238-005f28ee87e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.751955 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cac322-ff5c-445a-a238-005f28ee87e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.752237 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cac322-ff5c-445a-a238-005f28ee87e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.752512 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cac322-ff5c-445a-a238-005f28ee87e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.753977 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cac322-ff5c-445a-a238-005f28ee87e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.755543 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cac322-ff5c-445a-a238-005f28ee87e9 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.765619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1468/2927] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:04:53.769488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.779169 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1408-809a-4184-b674-fe2d5921c163 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.783742 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.785156 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.785436 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.785709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] PATH_INFO: `/v3/projects/a2b838ee426e479b8fd93939bdb5665d/users/bdf8b490be0d4f5e8ee4fa85c2800d3c/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.800691 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b838ee426e479b8fd93939bdb5665d, user_id=bdf8b490be0d4f5e8ee4fa85c2800d3c, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:53.803319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:53.815616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1460/2928] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:04:53 2022] PUT /identity/v3/projects/a2b838ee426e479b8fd93939bdb5665d/users/bdf8b490be0d4f5e8ee4fa85c2800d3c/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:04:53.825610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3daf7-bc51-4c10-9ee2-97faa422973c None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.825933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3daf7-bc51-4c10-9ee2-97faa422973c None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.826253 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3daf7-bc51-4c10-9ee2-97faa422973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:53.853584 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-62d3daf7-bc51-4c10-9ee2-97faa422973c None None] MFA Rules not processed for user `bdf8b490be0d4f5e8ee4fa85c2800d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:04:53.854791 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-62d3daf7-bc51-4c10-9ee2-97faa422973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:53.917164 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1469/2929] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:04:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:04:53.928461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:04:53.930158 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:04:53.989613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa9e74-77ef-45c2-8a50-39a6325b04ec admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:04:53.993442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2d48e-c257-46d6-87f3-52e9ee8aa02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31bf0e50e20b47d0a63662bbecce833d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:04:53.994426 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2d48e-c257-46d6-87f3-52e9ee8aa02e service cinder] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:04:53.994709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2d48e-c257-46d6-87f3-52e9ee8aa02e service cinder] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:04:53.994986 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2d48e-c257-46d6-87f3-52e9ee8aa02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:04:54.004706 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2d48e-c257-46d6-87f3-52e9ee8aa02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:04:54.006200 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2d48e-c257-46d6-87f3-52e9ee8aa02e service cinder] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:04:54.021741 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1461/2930] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:04:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:03.617692 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3daf7-bc51-4c10-9ee2-97faa422973c None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:03.630994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:03.682297 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3daf7-bc51-4c10-9ee2-97faa422973c None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:03.686827 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:03.687686 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:03.687968 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:03.688245 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] PATH_INFO: `/v3/users/bdf8b490be0d4f5e8ee4fa85c2800d3c` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:03.689462 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf8b490be0d4f5e8ee4fa85c2800d3c)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:03.691377 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:03.738568 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf8b490be0d4f5e8ee4fa85c2800d3c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:03.745759 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf8b490be0d4f5e8ee4fa85c2800d3c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:03.752569 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf8b490be0d4f5e8ee4fa85c2800d3c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:03.753216 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf8b490be0d4f5e8ee4fa85c2800d3c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:03.753826 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf8b490be0d4f5e8ee4fa85c2800d3c'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:03.770622 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1470/2931] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:05:03 2022] DELETE /identity/v3/users/bdf8b490be0d4f5e8ee4fa85c2800d3c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:05:03.775211 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:03.775664 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:03.775753 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:03.792924 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:03.794106 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:03.851907 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1462/2932] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:05:03.861607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:03.863610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:03.919300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06193e96-db95-4fad-a00d-e1c8de69921a admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:03.922926 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53ead5-7a5c-4f81-84d7-a4d4f6df6e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:03.923813 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53ead5-7a5c-4f81-84d7-a4d4f6df6e51 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:03.924095 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53ead5-7a5c-4f81-84d7-a4d4f6df6e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:03.924982 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53ead5-7a5c-4f81-84d7-a4d4f6df6e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:03.934109 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53ead5-7a5c-4f81-84d7-a4d4f6df6e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:03.935558 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53ead5-7a5c-4f81-84d7-a4d4f6df6e51 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:03.945190 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1471/2933] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:04.208005 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.209592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:04.221903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41587350-f7a6-410c-ab2c-3d679ef4f3e5 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.226424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.227289 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.227569 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.227845 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] PATH_INFO: `/v3/projects/a2b838ee426e479b8fd93939bdb5665d` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.229138 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b838ee426e479b8fd93939bdb5665d)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.230958 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.277897 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:04.281422 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b838ee426e479b8fd93939bdb5665d', 'request_id': 'req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:04.290429 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b838ee426e479b8fd93939bdb5665d', 'request_id': 'req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:04.296179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b838ee426e479b8fd93939bdb5665d', 'request_id': 'req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:04.307888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] The token cache is being invalidate because project a2b838ee426e479b8fd93939bdb5665d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80156) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:05:04.308177 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:04.308849 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-82389da5-cd01-4cf7-9ba2-ad05b621e0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:05:04.310399 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1463/2934] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:05:04 2022] DELETE /identity/v3/projects/a2b838ee426e479b8fd93939bdb5665d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:05:04.315942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fe108-af39-4699-8667-fff9ce814e66 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.316229 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fe108-af39-4699-8667-fff9ce814e66 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.316501 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fe108-af39-4699-8667-fff9ce814e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.335970 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6c8fe108-af39-4699-8667-fff9ce814e66 None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:04.337191 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6c8fe108-af39-4699-8667-fff9ce814e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:04.366349 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:04.396840 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1472/2935] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 10:05:04.405613 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fda850-6832-4a29-8dac-0581792654ed None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.405903 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fda850-6832-4a29-8dac-0581792654ed None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.406180 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fda850-6832-4a29-8dac-0581792654ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.424349 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-06fda850-6832-4a29-8dac-0581792654ed None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:04.425583 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-06fda850-6832-4a29-8dac-0581792654ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:04.433378 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1464/2936] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:05:04.437303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fe108-af39-4699-8667-fff9ce814e66 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.447329 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fe108-af39-4699-8667-fff9ce814e66 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.451930 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.452621 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.452911 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.453184 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] PATH_INFO: `/v3/domains` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.453897 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80157) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 10:05:04.454197 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.456336 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.467678 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1473/2937] 10.222.0.52 () {58 vars in 1152 bytes} [Fri Jun 24 10:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 10:05:04.471919 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fda850-6832-4a29-8dac-0581792654ed None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.481171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fda850-6832-4a29-8dac-0581792654ed None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.485822 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.486470 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.486770 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.487038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.487751 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.489682 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.515492 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1465/2938] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:05:04 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:05:04.519625 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.529532 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39921e-a876-42bf-a153-4b0b7ed4454f admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.584806 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.585463 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.585746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.586024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.586695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.588609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.607651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1474/2939] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:05:04 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 10:05:04.611731 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.621988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69431101-e029-4d35-8405-cb014fe4a435 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.626431 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.627076 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.627942 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.628216 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.628961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.630568 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.641006 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1466/2940] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:05:04.645369 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.655380 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be51d3d-4fdb-4e03-aab1-e41ec60e872e admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.659891 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0296c6-cd57-47f7-8b53-3eaee1d646c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.660534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0296c6-cd57-47f7-8b53-3eaee1d646c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.660867 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0296c6-cd57-47f7-8b53-3eaee1d646c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.661125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0296c6-cd57-47f7-8b53-3eaee1d646c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.661838 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0296c6-cd57-47f7-8b53-3eaee1d646c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.663510 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0296c6-cd57-47f7-8b53-3eaee1d646c3 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.673214 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1475/2941] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:05:04.678004 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.687763 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6cad7-f9f9-4e0b-aaaf-8eaaef9902b9 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.693021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.693888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] REQUEST_METHOD: `PUT` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.694174 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.694456 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] PATH_INFO: `/v3/projects/fff266b819494dfa90896b27839ff842/users/97affe47fd7343a19ab0a7972212d1b6/roles/0fc9d0c176a542f78847c87ede7b7b17` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.708709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff266b819494dfa90896b27839ff842, user_id=97affe47fd7343a19ab0a7972212d1b6, role_id=0fc9d0c176a542f78847c87ede7b7b17)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.711341 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.723987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1467/2942] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:05:04 2022] PUT /identity/v3/projects/fff266b819494dfa90896b27839ff842/users/97affe47fd7343a19ab0a7972212d1b6/roles/0fc9d0c176a542f78847c87ede7b7b17 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:05:04.728971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df7617-cace-42ce-a9c7-ef3e08bcecbb None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.729260 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df7617-cace-42ce-a9c7-ef3e08bcecbb None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.729538 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df7617-cace-42ce-a9c7-ef3e08bcecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.747548 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-44df7617-cace-42ce-a9c7-ef3e08bcecbb None None] MFA Rules not processed for user `c06586f3b93746d589ca131fe0aad76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:04.748763 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-44df7617-cace-42ce-a9c7-ef3e08bcecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:04.805546 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1476/2943] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:05:04.815586 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:04.817292 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:04.873223 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17da4-0807-438b-ae3a-bd1fe6e6f65f admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:04.876802 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:04.878062 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:04.878345 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:04.878622 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:04.887693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:04.889179 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:04.898577 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1468/2944] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:05:08.406213 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab692c1-3cd7-4680-8fc7-fcc4192f056e None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.406695 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab692c1-3cd7-4680-8fc7-fcc4192f056e None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.406787 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab692c1-3cd7-4680-8fc7-fcc4192f056e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.433641 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-6ab692c1-3cd7-4680-8fc7-fcc4192f056e None None] MFA Rules not processed for user `97affe47fd7343a19ab0a7972212d1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:08.434761 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-6ab692c1-3cd7-4680-8fc7-fcc4192f056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:08.466327 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:08.502933 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1477/2945] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:05:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:05:08.507443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.517115 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bbfa5-bb8d-480a-9d3b-2b416333da43 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.523164 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.523467 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.523756 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.524033 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.524745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.526629 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.556878 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1469/2946] 10.222.0.52 () {60 vars in 1152 bytes} [Fri Jun 24 10:05:08 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:05:08.558049 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab692c1-3cd7-4680-8fc7-fcc4192f056e None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.569032 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab692c1-3cd7-4680-8fc7-fcc4192f056e None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.633609 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.634300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.634689 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.634971 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.635731 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.637913 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.661201 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1478/2947] 10.222.0.52 () {60 vars in 1143 bytes} [Fri Jun 24 10:05:08 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 10:05:08.665193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.675824 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f4950-4fe1-4a19-9085-dd09089f9c2e admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.680419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.681152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.681437 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.681715 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.682461 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.684084 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.694920 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1470/2948] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 10:05:08.699059 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.709896 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf05071-a0c2-4e55-8e89-a160db4751e1 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.714578 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.715464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.715753 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.716033 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] PATH_INFO: `/v3/projects/ceddebfa7d7f4bcda816780b579365f3/users/631ebf06094c4a889ab94dce03381144/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.733055 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceddebfa7d7f4bcda816780b579365f3, user_id=631ebf06094c4a889ab94dce03381144, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.735788 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.750144 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1479/2949] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:05:08 2022] PUT /identity/v3/projects/ceddebfa7d7f4bcda816780b579365f3/users/631ebf06094c4a889ab94dce03381144/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:05:08.754214 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.787425 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437184-32e1-4ddf-ae0a-5ce89beb9f59 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.829548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:08.850893 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778c6dc-41da-48f1-9de4-abe3bce8902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.851605 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778c6dc-41da-48f1-9de4-abe3bce8902d admin admin] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.851887 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778c6dc-41da-48f1-9de4-abe3bce8902d admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.852171 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778c6dc-41da-48f1-9de4-abe3bce8902d admin admin] PATH_INFO: `/v3/roles` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.852939 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778c6dc-41da-48f1-9de4-abe3bce8902d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.854666 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778c6dc-41da-48f1-9de4-abe3bce8902d admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.870692 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1471/2950] 10.222.0.52 () {58 vars in 1121 bytes} [Fri Jun 24 10:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 10:05:08.872267 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:08.884471 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffed76e-8cce-47db-88c5-5da9f22bb2af admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:08.889420 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:08.890321 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:08.890607 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:08.891068 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] PATH_INFO: `/v3/projects/ceddebfa7d7f4bcda816780b579365f3/users/631ebf06094c4a889ab94dce03381144/roles/40cc7f5bdd5e4702924b038b7f946938` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:08.893746 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceddebfa7d7f4bcda816780b579365f3, user_id=631ebf06094c4a889ab94dce03381144, role_id=40cc7f5bdd5e4702924b038b7f946938)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:08.896414 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:08.908472 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1480/2951] 10.222.0.52 () {60 vars in 1482 bytes} [Fri Jun 24 10:05:08 2022] PUT /identity/v3/projects/ceddebfa7d7f4bcda816780b579365f3/users/631ebf06094c4a889ab94dce03381144/roles/40cc7f5bdd5e4702924b038b7f946938 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:05:13.235519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc3299-9e12-4726-851f-f1867e3ae869 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.236056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc3299-9e12-4726-851f-f1867e3ae869 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.236056 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc3299-9e12-4726-851f-f1867e3ae869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.263829 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-5fcc3299-9e12-4726-851f-f1867e3ae869 None None] MFA Rules not processed for user `631ebf06094c4a889ab94dce03381144`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:13.265015 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-5fcc3299-9e12-4726-851f-f1867e3ae869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:13.339477 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1472/2952] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:05:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:05:13.359004 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:13.366242 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:13.458520 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660aa206-ba0e-4df7-b8aa-c0f7fe8adecf admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:13.464316 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:13.464994 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:13.465278 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:13.465553 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:13.471332 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:13.526950 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:13.528545 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:13.538403 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1481/2953] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:14.585419 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:14.585888 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:14.585977 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:14.604125 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] MFA Rules not processed for user `97affe47fd7343a19ab0a7972212d1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:05:14.605400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:14.612319 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1473/2954] 10.222.0.52 () {58 vars in 957 bytes} [Fri Jun 24 10:05:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:05:23.534464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=25 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:24.598858 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=48 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:37.583729 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:37.646579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347205e-25ae-4ebf-808a-cf026ab02d7c service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:37.650441 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:37.651085 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:37.651373 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:37.651651 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:37.653456 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:37.675376 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:37.676829 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:37.685276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1482/2955] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:39.136906 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:39.138279 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:39.205650 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d42f69-2025-4926-8369-42d6bb55d618 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:39.210616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:39.211262 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:39.211550 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:39.211828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:39.213612 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:39.227488 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:39.229175 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:39.239417 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1474/2956] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:40.859073 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:40.868225 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7035c-7af4-4e5b-853e-5a225361ad2e service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:40.871944 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:40.872619 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:40.872903 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:40.873176 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:40.875013 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:40.885793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:40.887303 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:40.897224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1483/2957] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:41.570183 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:41.587680 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67d7f-d46b-4af3-b0b8-d094758d5b45 service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:41.601652 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:41.602327 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:41.602609 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:41.602884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:41.604833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:41.638449 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:41.640191 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:41.653278 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1475/2958] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:50.892282 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=41 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:51.647677 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=43 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:05:51.927564 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:51.929534 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:51.952125 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6130ee-92a7-4fe3-a5cb-251e9971b5ec service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:51.959363 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48647bb993344a8b91cec7f623aea007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:51.960221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:51.960508 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:51.961322 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:51.963547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:05:51.979069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:51.980710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:51.993571 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1484/2959] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:05:54.594984 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:05:54.604411 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005494f-27f9-4309-8b07-b59fa525121c service nova] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:05:54.608068 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403758f4-d3dc-4639-b582-026d13153b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:05:54.608709 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403758f4-d3dc-4639-b582-026d13153b23 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:05:54.608991 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403758f4-d3dc-4639-b582-026d13153b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:05:54.609264 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403758f4-d3dc-4639-b582-026d13153b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:05:54.618157 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403758f4-d3dc-4639-b582-026d13153b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:05:54.619564 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403758f4-d3dc-4639-b582-026d13153b23 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:05:54.628854 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1476/2960] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:05:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:06:01.987193 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=6 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:06:04.625152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=6 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:23.428069 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:23.429567 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:23.440938 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa7e94-34f4-4c44-a0ea-fb8ca83d6ceb service nova] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:23.444487 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845b0d7-1826-4d03-a6b2-6789c446af25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:23.445146 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b0d7-1826-4d03-a6b2-6789c446af25 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:23.445430 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b0d7-1826-4d03-a6b2-6789c446af25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:23.445707 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845b0d7-1826-4d03-a6b2-6789c446af25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:23.447440 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-a845b0d7-1826-4d03-a6b2-6789c446af25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:23.456682 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845b0d7-1826-4d03-a6b2-6789c446af25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:23.458088 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845b0d7-1826-4d03-a6b2-6789c446af25 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:23.466024 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1485/2961] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:23.566904 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f68df-a30c-41f9-9bd6-6c0d7cee0195 None None] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:23.567518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f68df-a30c-41f9-9bd6-6c0d7cee0195 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:23.567518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f68df-a30c-41f9-9bd6-6c0d7cee0195 None None] PATH_INFO: `/` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:23.568890 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1477/2962] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 10:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:23.575196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e796ea-30ab-4b3d-ba1f-41b2699f1eba None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:23.575485 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e796ea-30ab-4b3d-ba1f-41b2699f1eba None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:23.575767 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e796ea-30ab-4b3d-ba1f-41b2699f1eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:23.623082 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-67e796ea-30ab-4b3d-ba1f-41b2699f1eba None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:23.624391 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-67e796ea-30ab-4b3d-ba1f-41b2699f1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:23.632445 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1486/2963] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 10:08:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:23.644647 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f68df-a30c-41f9-9bd6-6c0d7cee0195 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:23.655595 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f68df-a30c-41f9-9bd6-6c0d7cee0195 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:23.659333 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003ae27-0bef-4418-943a-162661cef9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:23.659972 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ae27-0bef-4418-943a-162661cef9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:23.660254 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ae27-0bef-4418-943a-162661cef9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:23.660631 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ae27-0bef-4418-943a-162661cef9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:23.667936 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003ae27-0bef-4418-943a-162661cef9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:23.669420 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003ae27-0bef-4418-943a-162661cef9d6 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:23.678451 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1478/2964] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:08:23.766599 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0873-7f73-440e-a888-d4d4d6b2e6fe None None] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:23.766921 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0873-7f73-440e-a888-d4d4d6b2e6fe None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:23.767231 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0873-7f73-440e-a888-d4d4d6b2e6fe None None] PATH_INFO: `/` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:23.769196 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1487/2965] 10.222.0.52 () {54 vars in 921 bytes} [Fri Jun 24 10:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 10:08:23.774301 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] REQUEST_METHOD: `POST` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:23.774626 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:23.774902 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:23.815102 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.auth.core [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] MFA Rules not processed for user `48647bb993344a8b91cec7f623aea007`. Rule list: `[]` (Enabled: `True`). {{(pid=80156) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:08:23.816653 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:23.826217 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1479/2966] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Jun 24 10:08:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:08:23.837793 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a0873-7f73-440e-a888-d4d4d6b2e6fe None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:23.852160 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a0873-7f73-440e-a888-d4d4d6b2e6fe None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:23.855775 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dff4a-52e5-44cb-a61a-d3fca9185d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:23.856408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dff4a-52e5-44cb-a61a-d3fca9185d18 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:23.856843 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dff4a-52e5-44cb-a61a-d3fca9185d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:23.857221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dff4a-52e5-44cb-a61a-d3fca9185d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:23.869662 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dff4a-52e5-44cb-a61a-d3fca9185d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:23.871610 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dff4a-52e5-44cb-a61a-d3fca9185d18 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:23.882221 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1488/2967] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 10:08:33.812361 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=15 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:33.877594 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=25 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:08:39.450532 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:08:39.451934 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:39.478959 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21579b-7432-4971-82a1-0be869d7a578 None None] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:08:39.484021 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:08:39.484693 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:08:39.484982 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:08:39.485263 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:08:39.487014 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:08:39.498320 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:08:39.499884 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:08:39.510285 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1480/2968] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:08:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:08:49.506055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=11 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:05.640600 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d762-97bd-4681-8619-6d5c7da848f7 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:05.641195 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d762-97bd-4681-8619-6d5c7da848f7 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:05.641301 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d762-97bd-4681-8619-6d5c7da848f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:05.660660 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-3067d762-97bd-4681-8619-6d5c7da848f7 None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:05.662155 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-3067d762-97bd-4681-8619-6d5c7da848f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:05.669008 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1489/2969] 10.222.0.52 () {58 vars in 1061 bytes} [Fri Jun 24 10:10:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:05.675787 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:05.685094 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4f492-365d-4691-a1e0-11fa23eeae6e service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:05.688686 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:05.689355 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:05.689638 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:05.689917 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:05.691749 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:05.705291 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:05.706813 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:05.718012 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1481/2970] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:06.893713 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a53bc6-d527-4536-aeea-2311ee2ffe12 None None] REQUEST_METHOD: `POST` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:06.894276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a53bc6-d527-4536-aeea-2311ee2ffe12 None None] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:06.894276 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a53bc6-d527-4536-aeea-2311ee2ffe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:06.912084 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.auth.core [None req-81a53bc6-d527-4536-aeea-2311ee2ffe12 None None] MFA Rules not processed for user `ff6a58944beb49519f991a753adaa521`. Rule list: `[]` (Enabled: `True`). {{(pid=80157) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 10:10:06.913300 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-81a53bc6-d527-4536-aeea-2311ee2ffe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:06.919910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1490/2971] 10.222.0.52 () {58 vars in 1061 bytes} [Fri Jun 24 10:10:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 10:10:06.926226 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:06.934500 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483866f7-b5dd-412d-aab3-4e034d3ac29c service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:06.938055 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:06.938683 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:06.938961 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:06.939235 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:06.941604 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:06.954988 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:06.956443 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:06.967373 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1482/2972] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:10.685077 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a53bc6-d527-4536-aeea-2311ee2ffe12 None None] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:10.693706 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a53bc6-d527-4536-aeea-2311ee2ffe12 None None] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:10.697408 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff6a58944beb49519f991a753adaa521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:10.698034 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:10.698314 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:10.698588 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:10.757284 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:10.758710 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:10.768074 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1491/2973] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:13.525828 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:13.539517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765737a8-83b7-4c39-a627-91a27f99dc76 service neutron] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:13.547261 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:13.548058 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:13.548445 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:13.548857 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:13.551374 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:13.571833 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:13.573541 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:13.590144 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1483/2974] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:16.622189 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:16.634458 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c2abb-96f5-451a-8974-5238d3f9fb20 service neutron] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:16.639823 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:16.640464 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] REQUEST_METHOD: `GET` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:16.640781 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:16.641056 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:16.642824 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:16.659942 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:16.662064 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:16.676046 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1492/2975] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:10:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:10:16.684440 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:16.697424 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5ad7-8748-4f92-9c5b-6febab0a8d82 service placement] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:16.703132 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:16.704003 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:16.704283 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:16.704587 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] PATH_INFO: `/v3/users/97affe47fd7343a19ab0a7972212d1b6` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:16.705782 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97affe47fd7343a19ab0a7972212d1b6)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:16.708256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:16.755730 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97affe47fd7343a19ab0a7972212d1b6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:16.763270 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97affe47fd7343a19ab0a7972212d1b6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:16.771046 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97affe47fd7343a19ab0a7972212d1b6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:16.771674 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97affe47fd7343a19ab0a7972212d1b6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:16.772400 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97affe47fd7343a19ab0a7972212d1b6'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:16.789669 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1484/2976] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:10:16 2022] DELETE /identity/v3/users/97affe47fd7343a19ab0a7972212d1b6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:10:16.963574 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:16.975452 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2b432-197f-4576-a1f1-686a34a05ed3 service placement] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:17.032580 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:17.033496 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:17.033798 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:17.034175 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] PATH_INFO: `/v3/projects/fff266b819494dfa90896b27839ff842` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:17.035667 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff266b819494dfa90896b27839ff842)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:17.037623 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:17.082863 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff266b819494dfa90896b27839ff842', 'request_id': 'req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.090575 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff266b819494dfa90896b27839ff842', 'request_id': 'req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.096224 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff266b819494dfa90896b27839ff842', 'request_id': 'req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.103105 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:17.108850 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] The token cache is being invalidate because project fff266b819494dfa90896b27839ff842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:10:17.109304 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.110097 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.112603 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1493/2977] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:10:16 2022] DELETE /identity/v3/projects/fff266b819494dfa90896b27839ff842 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 10:10:17.116518 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:17.117987 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:17.131339 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c54eb-81da-4c36-b4fa-d3cad26a38b5 admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:17.187519 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:17.188442 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:17.188754 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:17.189030 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] PATH_INFO: `/v3/users/631ebf06094c4a889ab94dce03381144` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:17.190832 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=631ebf06094c4a889ab94dce03381144)` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:17.193090 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:17.231390 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:17.241745 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631ebf06094c4a889ab94dce03381144'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.248743 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631ebf06094c4a889ab94dce03381144'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.255616 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631ebf06094c4a889ab94dce03381144'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.256244 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631ebf06094c4a889ab94dce03381144'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.256861 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.notifications [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631ebf06094c4a889ab94dce03381144'} {{(pid=80156) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.274038 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1485/2978] 10.222.0.52 () {58 vars in 1223 bytes} [Fri Jun 24 10:10:17 2022] DELETE /identity/v3/users/631ebf06094c4a889ab94dce03381144 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:10:17.441199 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Authenticating user token {{(pid=80157) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:10:17.442716 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.fernet_utils [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80157) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:10:17.455328 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2adb83-a6dd-4b48-9754-f80a388bd3b6 admin admin] Validating token access rules against request {{(pid=80157) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:10:17.511676 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c06586f3b93746d589ca131fe0aad76b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccae62deda0a4d329baa9cb863e45e2d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80157) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:10:17.512628 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:10:17.512917 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:10:17.513200 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] PATH_INFO: `/v3/projects/ceddebfa7d7f4bcda816780b579365f3` {{(pid=80157) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:10:17.514547 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceddebfa7d7f4bcda816780b579365f3)` {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:10:17.516450 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] RBAC: Authorization granted {{(pid=80157) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:10:17.561579 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceddebfa7d7f4bcda816780b579365f3', 'request_id': 'req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.569307 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceddebfa7d7f4bcda816780b579365f3', 'request_id': 'req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.574639 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceddebfa7d7f4bcda816780b579365f3', 'request_id': 'req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8'} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.586087 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] The token cache is being invalidate because project ceddebfa7d7f4bcda816780b579365f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80157) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 10:10:17.586375 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.587006 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG keystone.notifications [None req-f75884d0-d4a0-4d04-9bac-bee37b9dd9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80157) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 10:10:17.588576 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: [pid: 80157|app: 0|req: 1494/2979] 10.222.0.52 () {58 vars in 1232 bytes} [Fri Jun 24 10:10:17 2022] DELETE /identity/v3/projects/ceddebfa7d7f4bcda816780b579365f3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 10:10:27.270592 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:10:27.582910 dv-cinder-845990-4-iscsi devstack@keystone.service[80157]: DEBUG dbcounter [-] [80157] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80157) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 10:13:44.413152 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Authenticating user token {{(pid=80156) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 10:13:44.414548 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:13:44.493515 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02d273-9993-4e9e-b46a-0af8bfe9b9bc admin admin] Validating token access rules against request {{(pid=80156) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 10:13:44.551272 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee1561-84bd-445d-be7c-e5c4d722ac7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505dda787dac43bca3595fa1c406a457', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f199023babeb4ffa88076256ba736ba1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80156) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 10:13:44.551912 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee1561-84bd-445d-be7c-e5c4d722ac7c service placement] REQUEST_METHOD: `GET` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 10:13:44.552193 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee1561-84bd-445d-be7c-e5c4d722ac7c service placement] SCRIPT_NAME: `/identity` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 10:13:44.552469 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee1561-84bd-445d-be7c-e5c4d722ac7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80156) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 10:13:44.554632 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.fernet_utils [None req-f0ee1561-84bd-445d-be7c-e5c4d722ac7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80156) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 10:13:44.565466 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee1561-84bd-445d-be7c-e5c4d722ac7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 10:13:44.567517 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee1561-84bd-445d-be7c-e5c4d722ac7c service placement] RBAC: Authorization granted {{(pid=80156) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 10:13:44.577256 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: [pid: 80156|app: 0|req: 1486/2980] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Jun 24 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 10:13:54.573051 dv-cinder-845990-4-iscsi devstack@keystone.service[80156]: DEBUG dbcounter [-] [80156] Writing DB stats keystone:SELECT=62 {{(pid=80156) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}